25: The Constructive Nature of Visual Processing

• Visual Perception Is a Constructive Process

• Visual Perception Is Mediated by the Geniculostriate Pathway

• Form, Color, Motion, and Depth Are Processed in Discrete Areas of the Cerebral Cortex

• The Receptive Fields of Neurons at Successive Relays in an Afferent Pathway Provide Clues to How the Brain Analyzes Visual Form

• The Visual Cortex Is Organized into Columns of Specialized Neurons

• Intrinsic Cortical Circuits Transform Neural Information

• Visual Information Is Represented by a Variety of Neural Codes

• An Overall View

Most of our impressions of the world and our memories of it are based on sight. Yet the mechanisms that underlie vision are not at all obvious. How do we perceive form and movement? How do we distinguish colors? Identifying objects in complex visual environments is an extraordinary computational achievement that artificial vision systems have yet to duplicate. Vision is used not only for object recognition but also for guiding our movements, and these separate functions are mediated by at least two parallel and interacting pathways.

The existence of parallel pathways in the visual system raises one of the central questions of cognition, the binding problem. How are different types of information carried by discrete pathways brought together into a coherent visual image?

Vision is often incorrectly compared to the operation of a camera. Unlike a camera, however, the visual system is able to create a three-dimensional representation of the world from the two-dimensional images on the retina. In addition, an object is perceived as the same under strikingly different visual conditions.

A camera reproduces point-by-point the light intensities in one plane of the visual field. The brain, in contrast, parses scenes into distinct components, separating foreground from background, to determine which light stimuli belong to one object and which to others. In doing so it uses previously learned rules about the structure of the world. In analyzing the incoming stream of visual signals the brain guesses at the scene presented to the eyes based on past experience.

This constructive nature of visual perception has only recently been fully appreciated. Earlier thinking about sensory perception was greatly influenced by the British empiricist philosophers, notably John Locke, David Hume, and George Berkeley, who thought of perception as an atomistic process in which simple sensory elements, such as color, shape, and brightness, were assembled in an additive way, component by component. The modern view that perception is an active and creative process that involves more than just the information provided to the retina has its roots in the philosophy of Immanuel Kant and was developed in detail in the early 20th century by the German psychologists Max Wertheimer, Kurt Koffka, and Wolfgang Köhler, who founded the school of Gestalt psychology.

The German term Gestalt means configuration or form. The central idea of the Gestalt psychologists is that what we see about a stimulus—the perceptual interpretation we make of any visual object—depends not just on the properties of the stimulus but also on its context, on other features in the visual field. The Gestalt psychologists argued that the visual system processes sensory information about the shape, color, distance, and movement of objects according to computational rules inherent in the system. The brain has a way of looking at the world, a set of expectations that derives in part from experience and in part from built-in neural wiring.

Max Wertheimer wrote: "There are entities where the behavior of the whole cannot be derived from its individual elements nor from the way these elements fit together; rather the opposite is true: the properties of any of the parts are determined by the intrinsic structural laws of the whole." In the early part of the 20th century the Gestalt psychologists worked out the laws of perception that determine how we see, including similarity, proximity, and good continuation.

We see a uniform six-by-six array of dots as either rows or columns because of the brain's tendency to impose a pattern. Thus if the dots in each row are similar we are more likely to see a pattern of alternating rows (Figure 25–1A). If the dots in each column are closer together than those in the rows, we are more disposed to see a pattern of columns (Figure 25–1B). The principle of good continuation is an important basis for linking line elements into unified shapes (Figure 25–1C). It is also seen in the phenomenon of contour saliency, whereby smooth contours tend to pop out from complex backgrounds (Figure 25–1D).

An important step in object recognition is separating figure from background. At different moments the same elements in the visual field can be organized into a recognizable figure or serve as part of the background for other figures (Figure 25–2). Segmentation relies not only on certain geometric principles, but also on cognitive influences such as attention and expectation. Thus a priming stimulus or an internal representation of object shape can facilitate the association of visual elements into a unified percept (Figure 25–3).

The brain analyzes a visual scene at three levels: low, intermediate, and high (Figure 25–4). At the lowest level, which we consider in the next chapter, visual attributes such as local contrast, orientation, color, and movement are discriminated. The intermediate level involves analysis of the layout of scenes and of surface properties, parsing the visual image into surfaces and global contours, and distinguishing foreground from background (see Chapter 27). The highest level involves object recognition (see Chapter 28). Once a scene has been parsed by the brain and objects recognized, the objects can be matched with memories of shapes and their associated meanings. Vision also has an important role in guiding body movement, particularly hand movement (see Chapter 29).

In vision as in other cognitive operations, various features—motion, depth, form, and color—occur together in a unified percept. This unity is achieved not by one hierarchical neural system but by multiple areas in the brain that are fed by at least two major interacting neural pathways. Because distributed processing is one of the main organizational principles in the neurobiology of vision, one must have a grasp of the anatomical pathways of the visual system to understand fully the physiological description of visual processing in later chapters.

In this chapter we lay the foundation for understanding the neural circuitry and organizational principles of the visual pathways. These principles apply quite broadly and are relevant not only for the multiple areas of the brain concerned with vision but also for other types of information processing by the brain.

Visual processing begins in the two retinae (see Chapter 26). The axons of the retinal ganglion cells, the projection neurons of the retina, form the optic nerve that extends to a midline crossing point, the optic chiasm. Beyond the chiasm fibers from the temporal hemiretinas proceed to the ipsilateral hemisphere; fibers from the nasal hemiretinas cross to the contralateral hemisphere (Figure 25–5). Because the temporal hemiretina of one eye sees the same half of the visual field (hemifield) as the nasal hemiretina of the other, the partial decussation of fibers at the chiasm ensures that all the information about each hemifield is processed in the visual cortex of the contralateral hemisphere.

Beyond the optic chiasm the axons from nasal and temporal hemiretinas carrying input from one hemifield join in the optic tract, which extends to the lateral geniculate nucleus of the thalamus. The lateral geniculate nucleus in primates consists of six layers, each of which receives input from either the ipsilateral or the contralateral eye. Because each layer contains a map of the contralateral hemifield, six concordant maps are stacked atop one another. The thalamic neurons then relay retinal information to the primary visual cortex.

The primary visual pathway is also called the geniculostriate pathway because it passes through the lateral geniculate nucleus on its way to the primary visual cortex, also known as the striate cortex because of the myelin-rich stripe that runs through its middle layers. A second pathway from the retina runs to the superior colliculus and is important in controlling eye movements. This pathway continues to the pontine formation in the brain stem and then to the extraocular motor nuclei. A third pathway extends from the retina to the pretectal area of the midbrain, where neurons mediate the pupillary reflexes that control the amount of light entering the eyes.

Each lateral geniculate nucleus projects to the primary visual cortex through a pathway known as the optic radiation (Figure 25–6A). These afferent fibers form a complete neural map of the contralateral visual field in the primary visual cortex. Beyond the striate cortex lie the extrastriate areas, a set of higher-order visual areas that are also organized as neural maps of the visual field. The preservation of the spatial arrangement of inputs from the retina is called retinotopy, and a neural map of the visual field is described as retinotopic or having a retinotopic frame of reference.

The primary visual cortex constitutes the first level of cortical processing of visual information. From there information is transmitted over two major pathways. A ventral pathway into the temporal lobe carries information about what the stimulus is, and a dorsal pathway into the parietal lobe carries information about where the stimulus is, information that is critical for guiding movement.

A major fiber bundle called the corpus callosum connects the two hemispheres, transmitting information across the midline. The primary visual cortex in either hemisphere represents slightly more than half the visual field, with the two hemifield representations overlapping at the vertical meridian. One of the functions of the corpus callosum is to unify the perception of objects spanning the vertical meridian by linking the cortical areas that represent opposite hemifields.

In the late 19th and early 20th centuries the cerebral cortex was differentiated by the anatomist Korbinian Brodmann and others using anatomical criteria. The criteria included the size, shape, and packing density of neurons in the cortical layers and the thickness and density of myelin. The functionally distinct cortical areas we have considered heretofore correspond only loosely to Brodmann's classification. The primary visual cortex (V1) is identical to Brodmann's area 17. In the extrastriate cortex the secondary visual area, V2, corresponds to area 18. Beyond that, however, area 19 contains several functionally distinct areas that generally cannot be defined by anatomical criteria.

The number of functionally discrete areas of visual cortex varies between species. Macaque monkeys have more than 30 areas. Although not all visual areas in humans have yet been identified, the number is likely to be at least as great as in the macaque. If one includes oculomotor areas and prefrontal areas contributing to visual memory, almost half of the cerebral cortex is involved with vision. Functional magnetic resonance imaging (fMRI) has made it possible to establish homologies between the visual areas of the macaque and human brains (Figure 25–7). Based on pathway tracing studies in monkeys, we now appreciate that these areas are organized in functional streams (Figure 25–7B).

The visual areas of cortex can be differentiated either by their representation of visual space, known as a visuotopic (or retinotopic) map, or by the functional properties of their neurons. Studies of these two differences have revealed that the visual areas are organized in two hierarchical pathways, a ventral pathway involved in object recognition and a dorsal pathway dedicated to the use of visual information for guiding movements. The ventral or object-recognition pathway extends from the primary visual cortex to the temporal lobe; it is described in detail in Chapter 28. The dorsal or movement-guidance pathway connects the primary visual cortex with the parietal lobe and then with the frontal lobes.

The pathways are interconnected so that information is shared. For example, movement information in the dorsal pathway can contribute to object recognition through kinematic cues. Information about movements in space derived from areas in the dorsal pathway is therefore important for the perception of object shape and is fed into the ventral pathway.

Reciprocity is an important feature of the connectivity between cortical areas. All connections between cortical areas are reciprocal—each area sends information back to the areas from which it receives input. These feedback connections provide information about cognitive functions, including spatial attention, stimulus expectation, and emotional content, to earlier levels of visual processing. The pulvinar in the thalamus serves as a relay between cortical areas (see Figure 25–6A).

The dorsal pathway courses through the parietal cortex, a region that uses visual information to direct the movement of the eyes and limbs, that is, for visuomotor integration. One area, the lateral intraparietal area named for its location in the interparietal sulcus, is involved in representing points in space that are the targets of eye movements or reaching. Patients with lesions of parietal areas fail to attend to objects on one side of the body, a syndrome called unilateral neglect (see Chapter 17).

The ventral pathway extends into the temporal lobe. The inferior temporal cortex stores information about the shapes and identities of objects; one portion represents faces, for damage to that region results in the inability to recognize faces (prosopagnosia) .

The dorsal and ventral pathways each comprise a hierarchical series of areas that can be delineated by several criteria. First, at many relays the array of inputs forms a map of the visual hemifield, and each such representation can be used to delineate a visual area. This is particularly useful at early levels of the pathway where the receptive fields of neurons are small and visuotopic maps are precisely organized. At higher levels, however, the receptive fields become larger, the maps less precise, and visuotopic organization is therefore a less reliable basis to delineate the boundaries of an area.

Another means to differentiate one area from another, as shown by experiments in monkeys, depends upon the distinctive functional properties exhibited by the neurons in each area. The clearest example of this is an area in the dorsal pathway, the middle temporal area (MT or V5), which contains neurons with a strong selectivity for the direction of movement across their receptive fields. Consistent with the idea that the middle temporal area is involved in the analysis of motion, lesions of this area produce deficits in the ability to track moving objects.

In 1906 Charles Sherrington coined the term receptive field in his analysis of the scratch withdrawal reflex: "The whole collection of points of skin surface from which the scratch-reflex can be elicited is termed the receptive field of that reflex." When it became possible to record from single neurons in the eye, H. Keffer Hartline applied the concept of the receptive field in his study of the retina of the horseshoe crab, Limulus: "The region of the retina which must be illuminated in order to obtain a response in any given fiber … is termed the receptive field of that fiber." In the visual system a neuron's receptive field represents a small window on visual space (Figure 25–8).

But responses measured with only one spot of light yielded only a limited understanding of a cell's receptive field. Using two small spots of light, both Hartline and Stephen Kuffler, who studied the mammalian retina, found an inhibitory surround or lateral inhibitory region in the receptive field. In 1953 Kuffler observed that "not only the areas from which responses can actually be set up by retinal illumination may be included in a definition of the receptive field but also all areas which show a functional connection, by an inhibitory or excitatory effect on a ganglion cell." Kuffler thus revealed that the receptive fields of retinal ganglion cells have functionally distinct subareas. These receptive fields have a center-surround organization and fall into one of two categories: on-center and off-center. Later work demonstrated that neurons in the lateral geniculate nucleus have similar receptive fields.

The on-center cells fire when a spot of light is turned on within a circular central region. Off-center cells fire when a spot of light in the center of their receptive field is turned off. The surrounding annular region has the opposite sign. For on-center cells a light stimulus that does not include the center produces a response when the light is turned off, a response termed on-center, off-surround. The center and surround areas are mutually inhibitory (Figure 25–9). When both center and surround are illuminated with diffuse light there is little or no response. Conversely, a light-dark boundary across the receptive field produces a brisk response. Because these neurons are most sensitive to borders and contours—to differences in illumination as opposed to uniform surfaces—they encode information about contrast in the visual field.

The size on the retina of a receptive field varies both according to the field's eccentricity —its position relative to the fovea, the central part of the retina where visual acuity is highest—and the position of neurons along the visual pathway. Receptive fields with the same eccentricity are relatively small at early levels in visual processing and become progressively larger at later levels. The size of the receptive field is expressed in terms of degrees of visual angle; the entire visual field covers nearly 180° (Figure 25–10A). In early relays of visual processing the receptive fields near the fovea are the smallest. The receptive fields for retinal ganglion cells that monitor portions of the fovea subtend approximately 0.1°, whereas those in the visual periphery reach up to 10°.

The amount of cortex dedicated to a degree of visual space changes with eccentricity. More cortical space is dedicated to the central part of vision, where the receptive fields are smallest and the visual system has the greatest spatial resolution (Figure 25–10C).

Receptive-field properties change from relay to relay along a visual pathway. By determining these properties one can assay the function of each relay nucleus and how visual information is progressively analyzed by the brain. For example, the change in receptive-field structure that occurs between the lateral geniculate nucleus and cerebral cortex reveals an important mechanism in the brain's analysis of visual form. The key property of the form pathway is selectivity for the orientation of contours in the visual field. This is an emergent property of signal processing in primary visual cortex; it is not a property of the cortical input but is generated within the cortex itself.

Whereas retinal ganglion cells and neurons in the lateral geniculate nucleus have concentric center-surround receptive fields, those in the cortex, although equally sensitive to contrast, also analyze contours. David Hubel and Torsten Wiesel discovered this characteristic in 1958 while studying what visual stimuli provoked activity in neurons in the primary visual cortex. While showing an anesthetized animal slides containing a variety of images, they recorded extracellularly from individual neurons in the visual cortex. As they switched from one slide to another they found a neuron that produced a brisk train of action potentials. The cell was responding not to the image on the slide but to the edge of the slide as it was moved into position.

The dominant feature of the functional organization of the primary visual cortex is the visuotopic organization of its cells: the visual field is systematically represented across the surface of the cortex (Figure 25–11A).

In addition, cells in the primary visual cortex with similar functional properties are located close together in columns that extend from the cortical surface to the white matter. The columns are concerned with the functional properties that are analyzed in any given cortical area and thus reflect the functional role of that area in vision. The properties that are developed in the primary visual cortex include orientation specificity and the integration of inputs from the two eyes, which is measured as the relative strength of input from each eye, or ocular dominance.

Ocular-dominance columns reflect the segregation of thalamocortical inputs arriving from different layers of the lateral geniculate nucleus. Alternating layers of this nucleus receive input from retinal ganglion cells located in either the ipsilateral or contralateral retina (Figure 25–12). This segregation is maintained in the inputs from the lateral geniculate nucleus to the primary visual cortex, producing the alternating left-eye and right-eye ocular dominance bands (Figure 25–11B), which receive input from the respective layers of the lateral geniculate nucleus.

Cells with similar orientation preferences are also grouped into columns. Across the cortical surface there is a regular clockwise and counterclockwise cycling of orientation preference with the full 180° cycle repeating every 750 μm (Figure 25–11C). One full cycle of orientation columns is called a hypercolumn. Likewise, the left- and right-eye dominance columns alternate with a periodicity of 750 to 1,000 μm. The orientation and ocular dominance columns are crisscrossed over the cortical surface.

Both types of columns were first mapped by recording the responses of neurons at closely spaced electrode penetrations in the cortex. The ocular-dominance columns were also identified by making lesions or tracer injections in individual layers of the lateral geniculate nucleus. More recently a technique known as optical imaging has enabled researchers to visualize a surface representation of the orientation and ocular dominance columns in living animals. Developed for studies of cortical organization by Amiram Grinvald, this technique visualizes changes in surface reflectance associated with the metabolic requirements of active groups of neurons, known as intrinsic-signal optical imaging, or changes in fluorescence of voltage-sensitive dyes. Intrinsic-signal imaging depends on activity-associated changes in local blood flow and alterations in the oxidative state of hemoglobin and other intrinsic chromophores.

An experimenter can visualize the distribution of cells with left or right ocular dominance, for example, by subtracting the image obtained while stimulating one eye from that acquired while stimulating the other. When viewed in a plane tangential to the cortical surface, the ocular dominance columns appear as alternating left- and right-eye stripes, each approximately 750 μm in width (Figure 25–11B).

The cycles of orientation columns form various structures, from parallel stripes to pinwheels. Sharp jumps in orientation preference occur at the pinwheel centers and "fractures" in the orientation map (Figure 25–11C). Superimposed on these is a third columnar system of continuously changing directional preference.

Embedded within the orientation and ocular-dominance columns are clusters of neurons that have poor orientation selectivity but strong color preferences. These units of specialization, located within the superficial layers, were revealed by a histochemical label for the enzyme cytochrome oxidase, which is distributed in a regular patchy pattern of blobs and interblobs. In the primary visual cortex these blobs are a few hundred micrometers in diameter and 750 μm apart (Figure 25–11D). The blobs correspond to clusters of color-selective neurons. Because they are rich in cells with color selectivity and poor in cells with orientation selectivity, the blobs are specialized to provide information about surfaces rather than edges.

In area V2 thick and thin dark stripes separated by pale stripes are evident with cytochrome oxidase labeling (Figure 25–11D). The thick stripes contain neurons selective for direction of movement and for binocular disparity as well as cells that are responsive to illusory contours and global disparity cues. The thin stripes hold cells specialized for color. The pale stripes contain orientation-selective neurons.

For every visual attribute to be analyzed at each position in the visual field there must be adequate tiling, or coverage, of neurons with different functional properties. As one moves in any direction across the cortical surface, the progression of the visuotopic location of receptive fields is gradual, whereas the cycling of columns occurs more rapidly. Any given position in space can therefore be analyzed adequately in terms of the orientation of contours, the color and direction of movement of objects, and the stereoscopic depth. The small segment of visual cortex that deals with that particular part of the visual field represents all possible values of all the columnar systems (Figure 25–13).

The columnar systems serve as the substrate for two fundamental types of connectivity along the visual pathway. Serial processing occurs in the successive connections between cortical areas, connections that run from the back of the brain forward. At the same time parallel processing occurs simultaneously in subsets of fibers that process different submodalities such as form, color, and movement.

Many areas of visual cortex reflect this arrangement; for example, functionally specific cells in V1 communicate with cells of the same specificity in V2. These pathways are not absolutely segregated, however, for there is some mixing of information between different visual attributes (Figure 25–14).

Columnar organization confers several advantages. It minimizes the distance required for neurons with similar functional properties to communicate with one another and allows them to share inputs from discrete pathways that convey information about particular sensory attributes. This efficient connectivity economizes on the use of brain volume and maximizes processing speed. The clustering of neurons into functional groups, as in the columns of the cortex, allows the brain to minimize the number of neurons required for analyzing different attributes. If all neurons were tuned for every attribute, the resultant combinatorial explosion would require a prohibitive number of neurons.

Each area of the visual cortex transforms information gathered by the eyes and processed at earlier synaptic relays into a signal that represents the visual scene. This transformation is accomplished by local circuits formed by excitatory and inhibitory neurons.

The principal input to the primary visual cortex comes from two parallel pathways that originate in the parvocellular and magnocellular layers of the lateral geniculate nucleus (see Figure 25–12). Neurons in the parvocellular layers project to cortical layer IVCβ, whereas those in the magnocellular layers project to layer IVCα. From there a sequence of interlaminar connections, mediated by the excitatory spiny stellate neurons, processes visual information over a stereotyped set of connections (Figure 25–15).

This characterization of parallel pathways is only an approximation, as there is considerable interaction between the pathways. This interaction is the means by which various visual features—color, form, depth, and movement—are linked, leading to a unified visual percept. One way this linkage, or binding, may be accomplished is through cells that are tuned to more than one attribute.

At each stage of cortical processing pyramidal neurons extend output to other brain areas. Superficial-layer cells are responsible for connections to higher-order areas of cortex. Layer V pyramidal neurons project to the superior colliculus and pons in the brain stem. Layer VI cells are responsible for feedback projections, both to the thalamus and to lower-order cortical areas.

Neurons in different layers have distinctive receptive-field properties. Neurons in the superficial layer of V1 have small receptive fields whereas deeper-layer neurons have large ones. The superficial-layer neurons are specialized for high-resolution pattern recognition. The deeper-layer neurons, such as those in layer V that are selective for the direction of movement, are specialized for the tracking of objects in space.

Feedback projections are thought to provide a means whereby higher centers in a pathway can influence lower ones. The number of neurons projecting from the cortex to the lateral geniculate nucleus is tenfold the number providing input to the cortex from the lateral geniculate nucleus. Although this feedback projection is obviously important, its function is largely unknown.

In addition to serial feed-forward and feedback connections, an important component of cortical circuits are the fibers that travel parallel to the cortical surface within each layer and provide long-range horizontal connections (Figure 25–16). These connections and their role in the functional architecture of cortex were analyzed by Charles Gilbert and Torsten Wiesel, who used intracellular recordings and dye injection to correlate anatomical features with cortical function. Because the visual cortex is organized visuotopically, the horizontal connections allow target neurons to integrate information over a relatively large area of the visual field and are therefore important in assembling the components of a visual image into a unified percept.

Integration can also be achieved by other means. The considerable convergence and divergence of connections at the synaptic relays of the afferent visual pathway imply that the receptive fields of neurons are larger and more complex at each successive relay and thus necessarily acquire an integrative function. Feedback connections may also support integration, both because of their divergence and because they originate from cells with larger receptive fields.

Individual neurons in a sensory pathway respond to a range of stimulus values. For example, a neuron in a color-detection pathway is not limited to responding to one wavelength but is instead tuned to a range of wavelengths. A neuron's response peaks at a particular value and tails off on either side of that value, forming a bell-shaped tuning curve with a particular bandwidth. Thus a neuron with a peak response at 650 nm and a bandwidth of 100 nm might give identical responses at 600 nm and 700 nm.

To be able to determine the wavelength from neuronal signals one needs at least two neurons representing filters centered at different wavelengths. Each neuron can be thought of as a labeled line in which activity signals a stimulus with a given value. When more than one such neuron fires, the convergent signals at the postsynaptic relay represent a stimulus with a wavelength that is the weighted average of the values represented by all the inputs.

A single visual percept is the product of the activity of a number of neurons operating in a specific combinatorial and interactive fashion called a population code. Population coding has been modeled in various ways. The most prevalent model is called vector averaging.

We can illustrate population coding with a population of orientation-selective cells, each of which responds optimally to a line with a specific orientation. Each neuron responds not just to the preferred stimulus but rather to any line that falls within a range of orientations described by a Gaussian tuning curve with a particular bandwidth. A stimulus of a particular orientation most strongly activates cells with tuning curves centered at that orientation; cells with tuning curves centered away from but overlapping that orientation are excited less strongly.

Each cell's preferred orientation, the line label, is represented as a vector pointing in the direction of that orientation. Each cell's firing is a "vote" for the cell's line label, and the cell's firing rate represents the weighting of the vote. The cell's signal can thus be represented by a vector pointing in the direction of the cell's preferred orientation with a length proportional to the strength of the cell's response. For all the activated cells one can calculate a vector sum with a direction that represents the value of the stimulus (Figure 25–17).

Another aspect of the population code is the variability of a neuron's response to the same stimulus. Repeated presentations of the same stimulus to a neuron sensitive to that stimulus will elicit a range of responses. The most sensitive part of a neuron's tuning curve lies not at the peak but along the flanks, where the tuning curve is steepest. Here small changes in the value of a stimulus produce the strongest change in response. Changes in stimulus value must, however, be sufficient to elicit a change in response that significantly exceeds the normal variability in the response of the neuron. One can compare that amount of change to the perceptual discrimination threshold. When many neurons contribute to the discrimination, the signal-to-noise ratio increases, a process known as probability summation, and the critical difference in stimulus value required for a significant change in neuronal response is less.

When the brain represents a piece of information, an important consideration is the number of neurons that participate in that representation. Although all information about a visual stimulus is present in the retina, the retinal representation is not sufficient for object recognition. At the other end of the visual pathway some neurons in the temporal lobe are selective for complex objects, such as faces. Can an individual cell represent something as complex as a particular face? Such a hypothetical neuron has been dubbed a "grandmother cell" because it would represent exclusively a person's grandmother, or a "pontifical cell" because it would represent the apex of a hierarchical cognitive pathway.

The nervous system does not, however, represent entire objects by the activity of single neurons. Instead some cells represent parts of an object and an ensemble of neurons represents an entire object. Each member of the ensemble may participate in different ensembles that are activated by different objects. This arrangement is known as a distributed code. Distributed codes can involve a few neurons or many. In any case, a distributed code requires complex connectivity between the cells representing a face and those representing the name and experiences associated with that person.

The foregoing discussion assumes that neurons signal information by their firing rate and their line labels. An alternative hypothesis is that the timing of action potentials itself carries information in a kind of Morse code. The code might be read from the synchronous firing of different sets of neurons over time. At one instant one group of cells might fire together followed by the synchronous firing of another group. Over a single train of action potentials a single cell could participate in many such ensembles. Whether sensory information is represented this way, and whether the nervous system carries more information than that represented by firing rate alone, is not known.

Visual perception involves an interaction between the retina, thalamic nuclei, and multiple areas of the cerebral cortex. The retina defines the limits of vision: the ability to resolve fine details, the discrimination of tiny movements, and the capacity to detect subtle contrasts and differences in the wavelength of reflected light.

The visual cortex acquires information coming from complex scenes, parses it into the surfaces and contours belonging to individual objects, and segments those objects from their background. This process involves a simultaneous analysis of local properties, such as orientation, direction of movement, and color, as well as the integration of these properties across space.

The mechanisms of visual processing may be observed in the receptive-field properties of individual neurons and in the functional organization of the cortex. Certain aspects of vision are analyzed in parallel by distinct pathways, one of which is involved in object recognition and another in visually guided movements.