A major long-term goal of learning research is to develop a model for the powerful learning mechanisms incorporated in the cerebral cortex of the brain. While many aspects of learning in the brain remain to be discovered, certain broad properties of its performance are well known. These include the capability of the brain to incrementally update its learned model with each new training stimulus and its ability to perform some tasks with as little as a single training exemplar (as when recognizing a new stimulus following a single exposure and then improving recognition performance with further exposures). Learning in one part of the input space does not produce any major degradation of performance in other parts. Of the currently proposed neural network learning methods, only those that perform some type of local interpolation seem capable of satisfying these constraints.
A biologically-plausible model of learning would need to develop a complete incremental learning method. It would be possible to start performing classification with very small numbers of inputs if the initial set of feature weights could be assigned by using weights from some similar previous task (for example, the recognition of a new person would initially be based on feature weights that have proved useful for recognizing other people). These weights would then need to be optimized incrementally rather than through a batch process such as conjugate gradient. One hypothetical model for implementing a variable kernel approach with neurons would be to initially assign a neuron to each new training experience. Each neuron would fire in proportion to its distance from a new input due to a Gaussian-shaped receptive field. The implementation of a variable kernel with a constant sum of neuron activations would require lateral inhibition between neurons at the same level, which is known to be a common aspect of cortical processing. To limit memory requirements, new inputs that are very similar to previous inputs would not be assigned to a new neuron, but instead would modify the output weights of the closest existing neurons to reflect the new output. This is similar to the role of the output layer of weights in RBF learning, so the learning would tend to switch from VSM to RBF approaches as the density of neighbours rose beyond what was needed to represent output variations. An open research problem is to derive a statistical test to determine when output variations are small enough to perform this combination of exemplars. One prediction that arises from VSM learning is that relative feature weights should be set on a more global basis than a single neuron (this differs from the separate feature weights of each unit in back-propagation). This could be accomplished in biological systems by determining feature weights from, for example, the activation level of a feature-encoding neuron rather than by changing individual synapse weights.