Table of contents

The PDF file below contains the full text of this editorial.

Internal models and adaptive controls are empirical and mathematical paradigms that have evolved separately to describe learning control processes in brain systems and engineering systems, respectively. This paper presents a comprehensive appraisal of the correlation between these paradigms with a view to forging a unified theoretical framework that may benefit both disciplines. It is suggested that the classic equilibrium-point theory of impedance control of arm movement is analogous to continuous gain-scheduling or high-gain adaptive control within or across movement trials, respectively, and that the recently proposed inverse internal model is akin to adaptive sliding control originally for robotic manipulator applications. Modular internal models' architecture for multiple motor tasks is a form of multi-model adaptive control. Stochastic methods, such as generalized predictive control, reinforcement learning, Bayesian learning and Hebbian feedback covariance learning, are reviewed and their possible relevance to motor control is discussed. Possible applicability of a Luenberger observer and an extended Kalman filter to state estimation problems—such as sensorimotor prediction or the resolution of vestibular sensory ambiguity—is also discussed. The important role played by vestibular system identification in postural control suggests an indirect adaptive control scheme whereby system states or parameters are explicitly estimated prior to the implementation of control. This interdisciplinary framework should facilitate the experimental elucidation of the mechanisms of internal models in sensorimotor systems and the reverse engineering of such neural mechanisms into novel brain-inspired adaptive control paradigms in future.

The ability to navigate in the world and execute appropriate behavioral and motor responses depends critically on our capacity to construct an accurate internal representation of our current motion and orientation in space. Vestibular sensory signals are among those that may make an essential contribution to the construction of such 'internal models'. Movement in a gravitational environment represents a situation where the construction of internal models becomes particularly important because the otolith organs, like any linear accelerometer, sense inertial and gravitational accelerations equivalently. Otolith afferents thus provide inherently ambiguous motion information, as they respond identically to translation and head reorientation relative to gravity. Resolution of this ambiguity requires the nonlinear integration of linear acceleration and angular velocity cues, as predicted by the physical equations of motion. Here, we summarize evidence that during translations and tilts from upright the firing rates of brainstem and cerebellar neurons encode a combination of dynamically processed semicircular canal and otolith signals appropriate to construct an internal model representation of the computations required for inertial motion detection.

Our sense of self-motion and self-orientation results from combining information from different sources. We hypothesize that the central nervous system (CNS) uses internal models of the laws of physics to merge cues provided by different sensory systems. Different models that include internal models have been proposed; we focus herein on that referred to as the sensory weighting model (Zupan et al 2002 Biol. Cybern.86 209–30). For simplicity, we isolate the portion of the sensory weighting model that estimates head angular velocity: it includes an inverse internal model of head kinematics and an 'idiotropic' vector aligned with the main body axis. Following a post-rotatory tilt in the dark, which is a rapid tilt following a constant-velocity rotation about an earth-vertical axis, the inverse internal model is applied to conflicting vestibular signals. Consequently, the CNS computes an inaccurate estimate of head angular velocity that shifts toward alignment with an estimate of gravity. Since reflexive eye movements known as vestibulo–ocular reflexes (VOR) compensate for this estimate of head angular velocity, the model predicts that the VOR rotation axis shifts toward alignment with this estimate of gravity and that the VOR time constant depends on final head orientation. These predictions are consistent with experimental data.

Prevailing views on how we time the interception of a moving object assume that the visual inputs are informationally sufficient to estimate the time-to-contact from the object's kinematics. However, there are limitations in the visual system that raise questions about the general validity of these theories. Most notably, vision is poorly sensitive to arbitrary accelerations. How then does the brain deal with the motion of objects accelerated by Earth's gravity? Here we review evidence in favor of the view that the brain makes the best estimate about target motion based on visually measured kinematics and an a priori guess about the causes of motion. According to this theory, a predictive model is used to extrapolate time-to-contact from the expected kinetics in the Earth's gravitational field.

The question of whether time is its own best representation is explored. Though there is theoretical debate between proponents of internal models and embedded cognition proponents (e.g. Brooks R 1991 Artificial Intelligence47 139–59) concerning whether the world is its own best model, proponents of internal models are often content to let time be its own best representation. This happens via the time update of the model that simply allows the model's state to evolve along with the state of the modeled domain. I argue that this is neither necessary nor advisable. I show that this is not necessary by describing how internal modeling approaches can be generalized to schemes that explicitly represent time by maintaining trajectory estimates rather than state estimates. Though there are a variety of ways this could be done, I illustrate the proposal with a scheme that combines filtering, smoothing and prediction to maintain an estimate of the modeled domain's trajectory over time. I show that letting time be its own representation is not advisable by showing how trajectory estimation schemes can provide accounts of temporal illusions, such as apparent motion, that pose serious difficulties for any scheme that lets time be its own representation.

The cerebellum evolved in association with the electric sense and vestibular sense of the earliest vertebrates. Accurate information provided by these sensory systems would have been essential for precise control of orienting behavior in predation. A simple model shows that individual spikes in electrosensory primary afferent neurons can be interpreted as measurements of prey location. Using this result, I construct a computational neural model in which the spatial distribution of spikes in a secondary electrosensory map forms a Monte Carlo approximation to the Bayesian posterior distribution of prey locations given the sense data. The neural circuit that emerges naturally to perform this task resembles the cerebellar-like hindbrain electrosensory filtering circuitry of sharks and other electrosensory vertebrates. The optimal filtering mechanism can be extended to handle dynamical targets observed from a dynamical platform; that is, to construct an optimal dynamical state estimator using spiking neurons. This may provide a generic model of cerebellar computation. Vertebrate motion-sensing neurons have specific fractional-order dynamical characteristics that allow Bayesian state estimators to be implemented elegantly and efficiently, using simple operations with asynchronous pulses, i.e. spikes. The computational neural models described in this paper represent a novel kind of particle filter, using spikes as particles. The models are specific and make testable predictions about computational mechanisms in cerebellar circuitry, while providing a plausible explanation of cerebellar contributions to aspects of motor control, perception and cognition.

We propose a model for human postural balance, combining state feedback control with optimal state estimation. State estimation uses an internal model of body and sensor dynamics to process sensor information and determine body orientation. Three sensory modalities are modeled: joint proprioception, vestibular organs in the inner ear, and vision. These are mated with a two degree-of-freedom model of body dynamics in the sagittal plane. Linear quadratic optimal control is used to design state feedback and estimation gains. Nine free parameters define the control objective and the signal-to-noise ratios of the sensors. The model predicts statistical properties of human sway in terms of covariance of ankle and hip motion. These predictions are compared with normal human responses to alterations in sensory conditions. With a single parameter set, the model successfully reproduces the general nature of postural motion as a function of sensory environment. Parameter variations reveal that the model is highly robust under normal sensory conditions, but not when two or more sensors are inaccurate. This behavior is similar to that of normal human subjects. We propose that age-related sensory changes may be modeled with decreased signal-to-noise ratios, and compare the model's behavior with degraded sensors against experimental measurements from older adults. We also examine removal of the model's vestibular sense, which leads to instability similar to that observed in bilateral vestibular loss subjects. The model may be useful for predicting which sensors are most critical for balance, and how much they can deteriorate before posture becomes unstable.

When the brain interacts with the environment it constantly adapts by representing the environment in a form that is called an internal model. The neurobiological basis for internal models is provided by the connectivity and the dynamical properties of neurons. Thus, the interactions between neural tissues and external devices provide a fundamental means for investigating the connectivity and dynamical properties of neural populations. We developed this idea, suggested in the 1980s by Valentino Braitenberg, for investigating and representing the dynamical behavior of neuronal populations in the brainstem of the lamprey. The brainstem was maintained in vitro and connected in a closed loop with two types of artificial device: (a) a simulated dynamical system and (b) a small mobile robot. In both cases, the device was controlled by recorded extracellular signals and its output was translated into electrical stimuli delivered to the neural system. The goal of the first study was to estimate the dynamical dimension of neural preparation in a single-input/single-output configuration. The dynamical dimension is the number of state variables that together with the applied input determine the output of a system. The results indicate that while this neural system has significant dynamical properties, its effective complexity, as established by the dynamical dimension, is rather moderate. In the second study, we considered a more specific situation, in which the same portion of the nervous system controls a robotic device in a two-input/two-output configuration. We fitted the input–output data from the neuro-robotic preparation to neural network models having different internal dynamics and we observed the generalization error of each model. Consistent with the first study, this second experiment showed that a simple recurrent dynamical model was able to capture the behavior of the hybrid system. This experimental and computational framework provides the means for investigating neural plasticity and internal representations in the context of brain–machine interfaces.

Studies of reaching suggest that humans adapt to novel arm dynamics by building internal models that transform planned sensory states of the limb, e.g., desired limb position and its derivatives, into motor commands, e.g., joint torques. Earlier work modeled this computation via a population of basis elements and used system identification techniques to estimate the tuning properties of the bases from the patterns of generalization. Here we hypothesized that the neural representation of planned sensory states in the internal model might resemble the signals from the peripheral sensors. These sensors normally encode the limb's actual sensory state in which movement errors occurred. We developed a set of equations based on properties of muscle spindles that estimated spindle discharge as a function of the limb's state during reaching and drawing of circles. We then implemented a simulation of a two-link arm that learned to move in various force fields using these spindle-like bases. The system produced a pattern of adaptation and generalization that accounted for a wide range of previously reported behavioral results. In particular, the bases showed gain-field interactions between encoding of limb position and velocity, very similar to the gain fields inferred from behavioral studies. The poor sensitivity of the bases to limb acceleration predicted behavioral results that were confirmed by experiment. We suggest that the internal model of limb dynamics is computed by the brain with neurons that encode the state of the limb in a manner similar to that expected of muscle spindle afferents.

Adaptive model theory (AMT) is a computational theory that addresses the difficult control problem posed by the musculoskeletal system in interaction with the environment. It proposes that the nervous system creates motor maps and task-dependent synergies to solve the problems of redundancy and limited central resources. These lead to the adaptive formation of task-dependent feedback/feedforward controllers able to generate stable, noninteractive control and render nonlinear interactions unobservable in sensory–motor relationships. AMT offers a unified account of how the nervous system might achieve these solutions by forming internal models. This is presented as the design of a simulator consisting of neural adaptive filters based on cerebellar circuitry. It incorporates a new network module that adaptively models (in real time) nonlinear relationships between inputs with changing and uncertain spectral and amplitude probability density functions as is the case for sensory and motor signals.

Recent behavioural and computational studies suggest that access to internal predictive models of arm and object dynamics is widespread in the sensorimotor system. Several systems, including those responsible for oculomotor and skeletomotor control, perceptual processing, postural control and mental imagery, are able to access predictions of the motion of the arm. A capacity to make and use predictions of object dynamics is similarly widespread. Here, we review recent studies looking at the predictive capacity of the central nervous system which reveal pervasive access to forward models of the environment.