33: The Organization and Planning of Movement

• Motor Commands Arise Through Sensorimotor Transformations

  • The Central Nervous System Forms Internal Models of Sensorimotor Transformations

  • Movement Inaccuracies Arise from Errors and Variability in the Transformations

  • Different Coordinate Systems May Be Employed at Different Stages of Sensorimotor Transformations

  • Stereotypical Patterns Are Employed in Many Movements

• Motor Signals Are Subject to Feedforward and Feedback Control

  • Feedforward Control Does Not Use Sensory Feedback

  • Feedback Control Uses Sensory Signals to Correct Movements

  • Prediction Compensates for Sensorimotor Delays

  • Sensory Processing Is Different for Action and Perception

• Motor Systems Must Adapt to Development and Experience

  • Motor Learning Involves Adapting Internal Models for Novel Kinematic and Dynamic Conditions

  • Kinematic and Dynamic Motor Learning Rely on Different Sensory Modalities

• An Overall View

In the preceding part of this book we considered how the brain constructs internal representations of the world around us. These internal representations have no intrinsic value and are behaviorally meaningful only when used to guide movement, whether foraging for food or attracting a waiter's attention. Thus the ultimate function of the sensory representations is to shape the actions of the motor systems. Sensory representations are the framework in which the motor systems plan, coordinate, and execute the motor programs responsible for purposeful movement.

In this part of the book we describe the principles of motor control that allow the brain and spinal cord to maintain balance and posture; to move our body, limbs, and eyes; and to communicate through speech and gesture.

Although movements are often classified according to function—eye movements, prehension (reach and grasp), posture, locomotion, breathing, and speech—many of these functions are subserved by overlapping groups of muscles. In addition, the same groups of muscles can be controlled voluntarily, rhythmically, or reflexively. For example, the muscles that control respiration can be used to take a deep breath voluntarily before diving under water, to breathe automatically and rhythmically in a regular cycle of inspiration and expiration, or to act reflexively in response to a noxious stimulus in the throat, producing a cough.

Voluntary movements are those that are under conscious control by the brain. Rhythmic movements can also be controlled voluntarily, but many such movements differ from voluntary movements in that their timing and spatial organization is to a large extent controlled autonomously by spinal or brain stem circuitry. Reflexes are stereotyped responses to specific stimuli that are generated by simple neural circuits in the spinal cord or brain stem. Although reflexes are highly adaptable to changes in behavioral goals, mainly because several different circuits exist to connect sensory and motor neurons, they cannot be directly controlled voluntarily.

In this chapter we focus on voluntary movements, using arm and hand movements to illustrate principles of sensorimotor control. Reflexes and rhythmic movements are discussed in detail in Chapters 35 and 36.

Conscious processes are not necessary for moment-to-moment control of movement. Although we may be aware of the intent to perform a task or of planning certain sequences of actions and at times are aware of deciding to move at a particular moment, movements generally seem to occur automatically. We carry out the most complicated movements without a thought to the actual joint motions or muscle contractions required. The tennis player does not consciously decide which muscles to use to return a serve with a backhand or which body parts must be moved to intercept the ball. In fact, thinking about each body movement before it takes place can disrupt the player's performance.

In this chapter we review the principles that govern the neural control of movement using concepts derived from behavioral studies and from computational models that are used both to understand the brain and to control the movements of robots. First, we look at how the brain transforms sensory inputs into motor outputs through a cascade of sensorimotor transformations. Second, we examine how sensory feedback can be used to correct errors that arise during movement. Finally, we see how motor learning allows us to improve our performance; to adapt to new mechanical conditions, as when using a tool; or to adapt to novel correspondences between sensory and motor events, for example when learning to use a computer mouse to control a cursor.

Motor outputs are neural commands that act on the muscles, causing them to contract and generate movement. These outputs are derived from sensory inputs in circuits that represent sensorimotor transformations. Sensory inputs include extrinsic information about the state of world as well as intrinsic information about our body. Extrinsic information, for example the spatial location of a target, can be provided by auditory and visual inputs. Intrinsic information includes both kinematic and kinetic information about our body.

Kinematic information includes the position, velocity, and acceleration of the hand, joint angles, and lengths of muscles without reference to the forces that cause them. Kinetic information is concerned with the forces generated or experienced by our body. These different forms of intrinsic information are provided by different sensors. For example, information about muscle lengths and their rate of change is provided mainly by muscle spindles, whereas Golgi tendon organs in the muscles and mechanoreceptors in our skin provide information about the force we are exerting.

Simple reflexes, such as a tendon-jerk reflex, involve a simple sensorimotor transformation: Sensory inputs cause motor output directly without the intervention of higher brain centers. However, voluntary movement requires multistage sensorimotor transformations. The involvement of multiple processing centers actually simplifies processing: Higher levels plan more general goals, whereas lower levels concern themselves with how these goals can be implemented.

Such a hierarchy accounts for the fact that a specific motor action, such as writing, can be performed in different ways with more or less the same result. Handwriting is structurally similar regardless of the size of the letters or the limb or body segment used to produce it (Figure 33–1). This phenomenon, termed motor equivalence, suggests that purposeful movements are represented in the brain abstractly rather than as sets of specific joint motions or muscle contractions. Such abstract representations of movement, able to drive different effectors, provide a degree of flexibility of action not practical with preset motor programs.

How do sensorimotor transformations generate movement to a desired location? For a person to reach toward an object, sensory information about the target's location must be converted into a sequence of muscle actions leading to joint rotations that will bring the hand to the target.

First, the target is localized in space relative to some part of the body such as the head or arm (egocentric space). Several sources of information are combined in this process. For example, the location of the target relative to the head is computed from the location of the target on each retina together with the direction of gaze of the eyes (Figure 33–2A). A person also needs to know the initial location of his hand or the tip of the tool that he wishes to place on the target (the end-effector or endpoint). The initial location of the endpoint can be estimated by combining visual inputs, proprioceptive signals, and tactile sensations, each of which can provide location information. Once the current configuration of the arm and location of the target are calculated, a movement can be planned. A plan typically has to specify both a particular path, the successive spatial positions of the endpoint, and also the trajectory, the time course over which these positions will be covered, and thus the accelerations and speeds of the movement (Figure 33–2B).

In a hierarchical model of planning the goal can be expressed in kinematic terms, such as the desired positions and velocities of the hand, or in kinetic terms, such as the force exerted by the hand. Movement can be planned as an endpoint trajectory, a desired change in the configuration of the limb expressed in coordinates intrinsic to the limb. Such a coordinate system could determine the change in joint angles or be based on a desired change in proprioceptive feedback. For example, the endpoint trajectory could be defined kinematically as the distance and direction the hand has to move to reach the target, as well as the speed along the path to the target.

Transformations can be expressed as changes in kinematic variables, such as the position of the hand and the joint angles that place the hand at that position. The calculation of an endpoint from a set of joint angles is termed forward kinematic transformation, whereas calculation of a set of joint angles that can reach an endpoint is termed inverse kinematic transformation (Figure 33–2C). This transformation must take into account the geometric parameters of the arm, such as the lengths of the upper arm and forearm (recall that kinematics considers motion without reference to the forces that cause it). The motor system controls joint angle by activating muscles that produce torques (rotational forces) at the joint.

The action of motor commands on muscles that results in a set of angular positions and velocities is known as the forward dynamic transformation. The term "dynamic" refers to the forces required to cause motion. However, to generate a desired joint angle trajectory the system must convert kinematic parameters into motor commands. That is, the system must calculate the torques at each joint necessary to achieve the motion and relate the force required to cause this motion to the desired acceleration of the limb. This transformation is known as the inverse dynamic transformation (Figure 33–2D). In general, to cause any acceleration the forces applied must exceed any resistive forces arising from the viscosity or stiffness of the limb, from gravity, and from external loads. The force not required to overcome the total resistive force will cause an angular acceleration, with the acceleration being dependant on the limb's inertia; the lower the inertia, the higher the acceleration.

Thus through a series of sensorimotor transformations, sensory input is finally converted into muscle contractions that generate movement. Although we have described one possible series of transformations that can achieve a movement, the actual computations used by the central nervous system are still under active investigation.

The Central Nervous System Forms Internal Models of Sensorimotor Transformations

We know from cellular studies that the central nervous system contains internal representations ("neural maps") of the various sensory receptor arrays and the musculature. Experimental and modeling studies strongly suggest that the central nervous system also maintains internal representations that relate motor commands to the sensory signals expected as a result of movement.

Given the fixed lengths of our limb segments, there is a mathematical relationship between the joint angles of the arm and the location of the hand in space. A neural representation of this relationship allows the central nervous system to estimate hand position if it knows the joint angles and segment lengths. The neural circuits that compute such sensorimotor transformations are examples of internal models (Box 33–1). Such neural representations may not exactly match true relationships because of structural differences (the models only approximate the true relationship between joint angles and hand position) or errors in the model's parameters (incorrect estimates of segment lengths).

An internal model that represents the causal relationship between actions and their consequences is called a forward model because it estimates future sensory inputs based on motor outputs. A forward model anticipates how the motor system's state will change as the result of a motor command. Thus, a copy of a descending motor command acting on the sensorimotor system is passed into a forward model that acts as a neural simulator of the musculoskeletal system moving in the environment. This copy of the motor command is known as an efference copy (or corollary discharge) to signify that it is a copy of the efferent signal flowing from the central nervous system to the muscles. We will see later how such simulations can be learned and used in sensorimotor control.

An internal model that calculates motor outputs from sensory inputs is known as an inverse model. Such a model can determine what motor commands are needed to produce the particular movements necessary to achieve a desired sensory consequence.

Forward and inverse models can be better understood if we place the two in series. If the structure and parameter values of each model are correct, the output of the forward model (the predicted behavior) will be the same as the input to the inverse model (the desired behavior) (Figure 33–3).

Movement Inaccuracies Arise from Errors and Variability in the Transformations

Motor control is often imprecise. Indeed, society celebrates those who can throw a dart into a small area of a board or hit a small white ball into a hole with a club. However, even the movements of the most skilled players show some degree of variability. In the 1890s the psychologist Robert Woodworth showed that fast movements are less accurate than slow ones. People slow their movements when accuracy is demanded. Inaccuracy can arise either from variability in the sensory inputs and motor outputs or from errors in the internal representations of this information.

An important component of sensorimotor variability is the intrinsic variability of our sensors and motor neurons because of fluctuations in their membrane potential. Because of these fluctuations, known as neural noise, the level of input signals required to trigger postsynaptic action potentials also varies. On the input side, neural noise limits the accuracy of estimates of the location of a target or limb (how near an estimate is to the true value) as well as their precision (how accurate the estimate is when repeated). On the output side, neural noise limits the accuracy and precision with which we contract our muscles. Moreover, the amount of noise in motor commands tends to increase with larger motor commands, limiting our ability to move rapidly and accurately at the same time. This increase in variability is caused by random variation in both the excitability of motor neurons and the recruitment of the additional motor units needed to produce increases in force.

Incremental increases in force are produced by progressively smaller sets of motor neurons, each of which produces disproportionately greater increments of force (see Chapter 34). Therefore, as force increases, fluctuations in the number of motor neurons lead to greater fluctuations in force. The consequences of this can be observed experimentally by asking subjects to generate a constant force or a force pulse of fixed amplitude. Not only are subjects unable to maintain constant force, but the variability of force also increases with the level of the force. Over a large range this increase in variability is captured by a constant coefficient of variation (the standard deviation divided by the mean force). This dependence of variability on force corresponds to the increase in the variability of pointing movements with the average speed of movement. The decrease in accuracy of movement with increasing speed is known as the speed-accuracy trade-off (Figure 33–4).

Errors can also arise from inaccuracy in the internal models that compute sensorimotor transformations. Neural representations of the musculature cannot easily capture the complex biomechanical properties of the musculoskeletal system, and this in turn significantly complicates the ability of the brain to compute accurate sensorimotor transformations. Indeed, the dependence of muscle force on the motor command is itself highly complex. A model prescribing motion in a system with just a single joint must not only estimate the muscle force (or torque) but also take into account inertia (the mass resisting acceleration), viscosity (resistive forces proportional to velocity), stiffness (elastic forces proportional to displacement) produced by the muscles and tendons opposing movement, and gravity.

The dynamic relationship between segments of limbs further complicates sensorimotor transformations. The motion of each segment produces torques, and potentially motions, at all other segments through mechanical interactions. For example, flexion of the upper arm through shoulder rotation can lead to either extension or flexion of the elbow depending on the initial elbow angle. In general, because of the interactions between linked segments, the torques needed to produce a specific change in angle at a particular joint depend not only on the muscles acting directly at this joint but also on the configurations and the motions of all other joints, and especially their acceleration. The brain develops an internal model of these complex mechanical interactions through learning early in childhood. We will see later that this learning is updated throughout life and depends critically on proprioception, which provides the brain with information about changes in muscle length and joint angles.

Different Coordinate Systems May Be Employed at Different Stages of Sensorimotor Transformations

Different coordinate systems are used in sensorimotor transformations and are encoded in several brain regions. Coordinate systems are either extrinsic or intrinsic to the body. Extrinsic coordinate systems relate objects in the outside world to other objects (allocentric coordinates) or to our body (egocentric coordinates) using exteroceptive information, usually visual or auditory (Figures 33–5A and B). Intrinsic coordinate systems, such as the set of muscle lengths or set of joint angles (Figure 33–5C), are based on information provided primarily by proprioceptive systems.

Elucidating the coordinate systems used in sensorimotor transformations is a major endeavor in neuroscience. We will see in later chapters that this issue can be fruitfully studied by examining how the firing patterns of neurons in different parts of the brain encode task features or movement parameters. Such studies aim to determine the variables (such as position or velocity) or type of coordinate system (such as allocentric or egocentric) that the neurons encode.

Behavioral studies also have used a variety of methods to examine the coordinate systems used in directing movement. One way has been to examine the details of the errors made during movement in different tasks. When subjects are asked to reach rapidly and repeatedly to a target, the error in the movements can be measured in different ways. If we average the final location of the hand across many trials, we may find a constant error or bias in the movement. We can examine the distribution of the final locations of the hand about this average position and infer from the patterns of constant and variable error the coordinate system used in the movement (Box 33–2).

Stereotypical Patterns Are Employed in Many Movements

Given a task, motor plans are underconstrained. For example, the hand can move to a target along an infinite number of possible paths, and for each path there is an infinite number of trajectories. Having specified the path and velocity, each point along the path could be reached by any number of combinations of arm joint angles and, owing to the overlapping actions of muscles and the ability to co-contract, each arm configuration could be achieved by many different combinations of muscles.

Although we have described different types of sensorimotor transformations, in general the inverse transformations cannot be uniquely specified. For example, the inverse kinematic transformation that transforms hand positions into joint angles can have many outputs based on the same input. This is because many different combinations of joint angles will put the hand in the same place. The ability of the motor systems to achieve a task in many different ways is called redundancy. If one way of achieving a task is not practical, there is usually an alternative.

Some of the earliest studies of movement examined how the brain determines the duration of a movement. Fitts's law describes the relationship between the amplitude, accuracy, and duration of a movement. This law relates the duration of a movement to the accuracy required of the task, as determined by the target width and amplitude of the movement, and applies to a variety of tasks such as reaching, placing pegs in holes, and picking up objects (Figure 33–8).

Despite variations in movement direction, speed, and location, several aspects of reaching movements are stereotypical or invariant. First, the hand tends to follow roughly a straight path (Figure 33–9A), although significant curvature is observed for certain movements, particularly vertical movements and movements near the boundaries of the reachable space. The tendency to make straight-line movements characterizes a large class of movements and is surprising given that the muscles act to rotate joints. Second, a plot of hand speed over time is typically smooth, unimodal, and roughly symmetric (bell-shaped) (Figure 33–9B). This is not the case when movement accuracy requirements are high or corrections to the movement are made.

In contrast, the motions of the joints in series (such as the shoulder, elbow, and wrist) are complicated and vary greatly with different initial and final positions. Because rotation at a single joint would produce an arc at the hand, both elbow and shoulder joints must be rotated concurrently to produce a straight path. In some directions the elbow moves more than the shoulder; in others, the reverse occurs. When the hand is moved from one side of the body to the other (see Figure 33–9A, movement from T2 to T5) one or both joints may have to reverse direction in midcourse. The fact that hand trajectories are more invariant than joint trajectories suggests that the motor system typically controls the hand by adjusting joint rotations and torques to achieve desired hand trajectories.

Invariances can also be seen in more complex movements. The nervous system puts together complex actions from elemental movements that have highly stereotyped spatial and temporal characteristics. For example, the seemingly continuous motion of drawing a figure eight actually consists of several discrete movements that are roughly constant in duration, regardless of their size. Moreover, there is a relationship between the curvature of each elemental movement and speed: Subjects tend to slow the hand as the curvature of the path increases. Empirical studies have shown that for many tasks a power law relation, the two-thirds power law, governs the relationship between hand speed and path curvature (Figure 33–10).

The simple spatiotemporal elements of a complex movement are called movement primitives or movement schemas. Like the simple lines, ovals, or squares in computer graphics programs, movement schemas can be scaled in size or time. The neural representations of complex actions, such as prehension, writing, typing, or drawing, are thought to be stored sets of these simple spatiotemporal elements.

Recent computational studies of a variety of tasks suggest that a repertory of movement schemas is the result of a process in which all possible ways of moving are ranked and the best is selected. This idea implies that either through evolution or motor learning our movements improve progressively until some limit is reached.

To quantify how good or bad a movement the brain assigns a cost to each possible movement, and the movement with the lowest cost is executed. The cost is specified as some function of the movement and task, and the challenge to researchers has been to determine, from observed movement patterns and perturbation studies, the form of this function.

The cost may be kinematic; for example, lack of smoothness in a movement can be corrected by minimizing the rate of change of hand acceleration summed over a movement. Alternatively, the cost may be dynamic. For example, because the variability in the motor output is proportional to the magnitude of the motor command, repetition of the same sequence of intended motor commands many times will lead to a distribution of actual movements. Modifying the sequence of motor commands can control aspects of this distribution, such as the spread of positions of the hand at the end of the movement. In a simple aiming movement the cost is the final error, as measured by the variability about the target. A model that minimizes this cost would accurately predict the trajectories of both eye and arm movements.

So far we have focused on how sensory inputs are used to plan a movement and why the resulting movements can have errors. Sensory inputs to the motor systems during a movement provide information about errors that arise from neural noise, from inaccuracies in the motor commands as a result of flaws in the internal models, or from changes in the outside world, such as the unexpected motion of a target. We now examine what part these errors play in two forms of motor control, feedforward and feedback control.

Feedforward Control Does Not Use Sensory Feedback

Movements that are not correctible during the movement are often termed ballistic. This term is ordinarily applied to the trajectory of unpowered projectiles (such as ballistic missiles) that, once launched, can no longer be controlled and are subject only to gravity. Because arm movements can potentially be controlled throughout their course, however, the term feedforward control more accurately describes the trajectory. Feedforward commands are generated without regard to the consequences. Such commands are also termed open-loop because the sensorimotor loop is not completed by sensory feedback (Figure 33–11A).

Open-loop control is advantageous if we consider the delays inherent in the sensorimotor system. Both conversion of stimulus energy into neural signals by stimulus receptors and conveyance of the sensory signals to central neurons take time. For example, visual feedback can take approximately 100 ms to be processed in the retina and transmitted to the visual cortex. In addition to delays in the peripheral sensory system, there are also delays in central processing, in the transmission of efferent signals to motor neurons, and in the response of muscles. In all, the combined sensorimotor loop delay is appreciable, approximately 200 ms for a response to a visual stimulus. This delay means that rapid movements, such as the saccades of the eye that last less than 100 ms, cannot use sensory feedback to guide the eye onto a target.

Even with slower movements, such as deliberate reaching, which can take approximately 500 ms, sensory information cannot be used to guide the initial part of a movement, so open-loop control must be used. This open-loop component can be clearly demonstrated. Both the initial speed and the acceleration of the hand during reaching are proportional to the distance of the target. This and the straightness of hand paths mean that the extent of a movement is planned before the movement is initiated, and the movement is generated in a feedforward manner (Figure 33–12).

Open-loop control also has disadvantages. Any movement errors caused by inaccuracies in planning or execution will not be corrected, and therefore will compound themselves over time or successive movements. The more complex the system under control, the more difficult it is to arrive at a perfect inverse model through learning. For example, the vestibulo-ocular reflex (see Chapter 40) uses open-loop control to maintain fixation during head rotation. This is a very efficient system as the dynamics of the eye are relatively simple, and the rotation of the head can be directly sensed by the vestibular labyrinth. The complexity of the arm, however, makes it very difficult to optimize an inverse model, and thus the control of hand movement requires some form of error correction.

Feedback Control Uses Sensory Signals to Correct Movements

To correct movement errors as they arise, the action must be monitored before it is completed. Such error-correcting systems are known as feedback or closed-loop systems because the sensorimotor loop is complete (Figure 33–11B).

The simplest form of feedback control is one in which the control system generates a fixed response when the error exceeds some threshold. Such a system is seen in most central heating systems in which a thermostat is set to a desired temperature. When the house temperature falls below the specified level, the heating is turned on until the temperature reaches that level. Although such a system is simple and can be effective, it has the drawback that the amount of heat being put into the house does not relate to the discrepancy between the actual and desired temperature (the error). A better system is one in which the control signal is proportional to the error.

Such proportional control of movement involves sensing the error between the actual and desired position of, for example, the hand. The size of the corrective motor command is in proportion to the size of the error and in a direction to reduce the error. The amount by which the corrective motor command is increased or decreased per unit of positional error is called the gain (Figure 33–13). By continuously correcting a movement, feedback control can be robust both to noise in the sensorimotor system and to environmental perturbations.

In most motor systems movement control is achieved through both feedforward and feedback processes. Because sensory feedback is not available for the first portion of a movement, feedforward processes generate the initial motor command only. As the movement progresses, information on performance becomes available, allowing feedback control to play a role.

When lifting an object between thumb and index finger, for example, sufficient grip force (perpendicular force between the digits and the object's surface) must be generated to prevent slippage owing to load force (tangential force between the digit and object surface arising from the object's weight). We use feedforward control to set our grip force and the lifting force in accordance with the expected slipperiness and weight of the object. If cutaneous receptors indicate that slippage is occurring, our grip force is increased immediately through rapid feedback control (Figure 33–14). Because cutaneous information on slippage evokes a motor command to increase grip force only when the object is being lifted, this feedback circuit is said to be "gated" during lifting.

Feedback control cannot generate a command in anticipation of an error: It is always driven by an error. Feedforward control, conversely, is based only on a desired state and can therefore null the error.

Prediction Compensates for Sensorimotor Delays

Accurate feedback control of movement requires information on the body's current state, for example, the positions and velocities of our body segments. However, sensory feedback from the periphery is both noisy and slow. Delays in feedback can lead to problems during a movement, as the delayed information does not reflect the present state of the body and world. Two strategies can compensate for such delays and thus increase the accuracy of sensory feedback during movement: intermittency of movement and prediction of changes in body states due to movement. With intermittency, movement is momentarily interrupted by rest, as in eye saccades and manual tracking. Provided the interval of rest is greater than the time delay of the sensorimotor loop, intermittency fosters more accurate sensory feedback.

Prediction is a better strategy and can form a major component of a state estimator. Although sensory signals provide necessary information about the body, the motor command also provides useful information. If both the current state of the body and the descending motor command are known, the next state of the body can be estimated. This estimate is derived from a forward model that predicts how the body will change in response to the motor command. Because this estimate is predictive, it is time-advanced, thereby compensating for the feedback delays. However, this estimate will tend to drift over time if the forward model is not perfectly accurate.

The drawbacks of using only sensory feedback or only motor prediction can be ameliorated by monitoring both and using a forward model to estimate the current state of the body. A neural apparatus that does this is known as an observer model. The major objectives of the observer model are to compensate for sensorimotor delays and to reduce uncertainty in the estimate of the state of the body owing to noise in both the sensory and motor signals (Figure 33–15). Such a model has been supported by empirical studies of how the nervous system estimates hand position, posture, and head orientation.

The nervous system has several different internal models of control that use prediction and sensory feedback to different extents. The comparative advantages of these various models are nicely illustrated by differences in object manipulation under different conditions. When the object's behavior is unpredictable, sensory feedback provides the most useful signal for estimating load. For example, when flying a kite we need to adjust our grip almost continuously in response to unpredictable motions of the kite. When dealing with such unpredictability, grip force needs to be high to prevent slippage because it tends to lag behind load force (Figure 33–16A).

However, when handling objects with stable properties, predictive control mechanisms can be effective. For example, when the load is increased by a self-generated action, such as moving the arm, the grip force increases instantaneously with load force (Figure 33–16B). Sensory detection of the load would be too slow to account for this rapid increase in grip force. Such predictive control is essential for the rapid movements commonly observed in dexterous behavior.

The discrepancy between actual and predicted sensory feedback is also essential in motor control. For example, when we pick up an object, we anticipate when the object will lift off. The brain is particularly sensitive to the occurrence of unexpected events or the nonoccurrence of an expected event. Thus if an object is lighter or heavier than expected, and therefore is lifted too early or cannot be lifted, reactive responses are evoked. The brain seems to pay particular attention to these critical moments to determine whether the subsequent actions that are part of the task should proceed.

In addition to its use in compensating for sensory feedback delays, prediction is a key element in perceptual processing. Sensory feedback can originate from two sources: either external sources or our own movement. In the sensory receptor these two sources are not distinguished, however, and sensory signals do not carry a label "external stimulus" or "internal stimulus."

Sensitivity to external events can be amplified by reducing the feedback from our own movement. Thus predictions of sensory signals that arise from our own movements are subtracted from the total sensory feedback, thereby enhancing the signals that carry information about external events. Such a predictive mechanism is responsible for the fact that tickling oneself is a less intense experience than tickling by another. When participants are asked to tickle themselves with a time delay introduced between the motor command and the resulting tickle, the greater the time delay the more ticklish the sensation. As the time delay increases, the predictor becomes more inaccurate, thereby failing to cancel the sensory feedback resulting in the tickle sensation.

Sensory Processing Is Different for Action and Perception

A growing body of research supports the idea that the sensory information used to control actions is processed in neural pathways that are distinct from the afferent pathways that contribute to perception. Mel Goodale and David Milner have proposed that visual information flows in two streams in the brain (see Chapter 25). A dorsal stream projects to the posterior parietal cortex and is particularly involved in the use of vision for action (see Chapter 38). Conversely, a ventral stream projects to the inferotemporal cortex and is involved in conscious visual perception (see Chapter 28).

This distinction between the uses of vision for action and perception is based on a double-dissociation seen in patient studies. For example, the patient D. F. developed visual agnosia after damage to her ventral stream. She is unable, for example, to explain the orientation of a slot either verbally or with her hand. However, when asked to perform a simple action, such as putting a card through the slot, she has no difficulty orienting her hand appropriately to put the card through the slot. Conversely, patients with damage to the dorsal stream can develop optic ataxia in which perception is intact, but control is affected.

Although the distinction between perception and action arose from clinical observations, it can also be seen in normal people, as in the size–weight illusion. When lifting two objects of different size but equal weight, people report that the smaller object feels heavier. This illusion, first documented more than 100 years ago, is both powerful and robust. It does not lessen when a person is informed that the objects are of equal weight and does not weaken with repeated lifting.

When subjects begin to lift large and small objects that weigh the same, they generate larger grip and load forces for the larger object because they assume that larger objects are heavier. After alternating between the two objects, they rapidly learn to scale their fingertip forces precisely for the true object weight (Figure 33–17). This shows that the sensorimotor system recognizes that the two weights are equal. Nevertheless, the size–weight illusion persists, suggesting not only that the illusion is a result of high-level cognitive centers in the brain but also that the sensorimotor system can operate independently of these centers.

Animals have a remarkable capacity for learning new motor skills through their interaction with the environment. This learning is distinct from and independent of the development of skills through maturation. Although evolution can hard-wire some motor behaviors, such as the ability of a foal to stand, motor behavior in general must adapt to new and varying environments.

New motor skills cannot be acquired by a fixed neural control system. Sensorimotor control systems must constantly adapt over a lifetime as body size and proportions change, thereby maintaining an appropriate relationship between motor commands and body mechanics. In addition, learning is the only way to acquire motor skills that are defined by social convention, such as writing or dancing.

Most forms of motor learning involve procedural or implicit learning, so-called because subjects are generally unable to express what it is they have learned. Implicit learning often takes place without consciously thinking about it and can be retained for extended periods of time without practice (see Chapter 66). Typical examples of procedural learning are learning to ride a bicycle or play the piano. In contrast, explicit or declarative learning involves the acquisition of knowledge that can be expressed in statements about the world and is available to introspection (see Chapter 67). Memorizing the names and routes of the cranial nerves or directions to the local hospital are examples of explicit learning. Declarative memory tends to be easily forgotten, although repeated exposure can lead to long-lasting retention.

Motor learning can occur more or less immediately or require some time. One learns to pick up an object of unknown weight almost immediately and learns to ride a bicycle after a little practice, but mastering the piano requires years. These different timescales may reflect the intrinsic difficulty of the task as well as evolutionary constraints that have to be unlearned to perform the task. For example, piano playing requires learning precise control of the fingers, whereas in normal movements, such as reaching and grasping, individuated finger movements are rare.

Motor Learning Involves Adapting Internal Models for Novel Kinematic and Dynamic Conditions

Sensorimotor transformations have kinematic and dynamic components. Kinematic transformations relate events in different spatial coordinate systems, such as joint angles of the arm and the position of the hand in space. To control a computer mouse, for example, we must learn the kinematic transformation between the handheld mouse and the image of the cursor on the screen. Dynamic transformations relate forces acting at the joints to the motion of the system. We must relate the forces we apply to the mouse to the resulting movement, a transformation that depends on the inertia of the mouse and the friction between the mouse and pad. The kinematics and dynamics of movement vary greatly as we grow and interact with new objects. The brain adapts by reorganizing or adjusting motor commands to generate new actions.

As we saw earlier, we normally move the hand in a straight line to reach an object. Unexpected dynamic interactions may produce curved paths, but subjects learn to anticipate these effects. This learning is conveniently studied by having subjects make pointing movements with an apparatus through which novel forces can be applied to the arm (Figure 33–18A). For example, applying a force that is proportional to the speed of the hand but which acts at right angles to the direction of movement forces the hand into a brief curving movement before reaching the target. Over time the subject adapts to this perturbation and is able to maintain a straight-line movement (Figure 33–18D).

Subjects might adapt to such a situation in two possible ways. First, they could co-contract the muscles in their arm, thereby stiffening the arm and reducing the impact of the perturbation. Alternatively, they could learn an internal model that compensates for the expected forces, one that uses a new set of motor commands. By examining the subjects' movements after the force is turned off, we can distinguish between these two forms of learning. If the arm simply stiffens, it should continue to move in a straight path. If a new internal model is learned, the new model should compensate for a force that no longer exists, thereby producing a path in the direction opposite from the earlier perturbation. In fact, when the force is turned off, subjects show a large after-effect in the opposite direction, demonstrating that they had learned to compensate for the perturbation (Figure 33–18D).

Although motor learning often takes much practice, once a task is no longer performed de-adaptation is typically quite swift. The context of the movement, that is the sensory inputs associated with a particular task, can be enough to switch behavior. When subjects wear prismatic glasses that rotate visual space, for example, they initially misreach targets but soon learn to reach correctly. After repeated trials the contextual cue of the feel of the glasses, without the prisms in place, is sufficient to switch subjects into behavior suitable with the prisms.

Kinematic and Dynamic Motor Learning Rely on Different Sensory Modalities

Not all sensory modalities are equally important in learning motor tasks. In learning dynamic tasks, proprioception is more important than vision. We normally learn dynamic tasks equally well with or without vision. Patients who have lost proprioception have particular difficulty controlling the dynamic properties of their limbs (Box 33–3) or learning new dynamic tasks without vision.

However, the same patients are easily able to adapt to drastic kinematic changes, such as tracing a drawing while viewing their hand in a mirror. In fact these subjects perform better than normal subjects at such a task, perhaps because they have learned to guide their movements visually and, because of the lack of proprioception, do not experience any conflict between vision and proprioception.

The primary purpose of the elaborate information processing and storage that occurs in the brain is to enable us to interact with our environment. Our infinitely varied and purposeful motor behaviors are governed by the integrated actions of the brain's several motor systems.

To control action the central nervous system uses a sequence of sensorimotor transformations that convert incoming sensory information into motor outputs. The brain uses internal models at each stage in the sensorimotor transformation. Variability in the inputs and outputs of these transformations and inaccuracies in their representation underlie the errors and variability in movement and lead to the ubiquitous trade-off between speed and accuracy.

The motor systems generate commands using feedforward circuits or error-correcting feedback circuits; most movement involves both types of control. The adverse effects of delays in feedback are reduced through the use of predictive processes.

Finally, motor control circuits are not static but undergo continual modification and recalibration throughout life. Motor learning improves motor control in novel situations, and different forms of sensory information are vital for learning.

The ease with which we conduct ordinary movements masks the true complexity of the control processes involved. Many factors inherent in sensorimotor control are responsible for this complexity, which becomes clearly evident when we try to build machines that can perform human-like control of movement. Although computers can beat grandmasters at chess, no computer can yet control a robot to manipulate a chess piece with the dexterity of a six-year-old child.