Kohonen Networks
According to Teuvo Kohonen, Kohonen Networks are a novel self-organizing mechanism. The main finding is that signals from a primary event space are automatically mapped onto a set of output responses in a basic network of adaptive physical elements. This mapping allows the responses to acquire the same topological order as the primary events. This indicates that the technique makes it easier for topologically accurate maps of observable event features to be automatically created.
As long as their feature values or signal representations can be described in a metric or topological space that permits their ordering, these networks can create maps of conceptual or abstract objects.
Retinotectal mapping is one example of a direct topographic projection that is thought to be generalized by this principle. Like perceptrons, which are also adaptive layered networks, Kohonen Networks processing units can be recognized by their tangible, physical adaptable components. The incorporation of local feedback between adjacent units, which is crucial for map development, is a distinguishing trait of these novel models. Assuming that the behavior of the system is properly described both structurally and functionally, the process depends only on external signal activity.
Structure and Architecture
Typically, a Kohonen Networks is organized as a two-dimensional or one-dimensional array of processing units. An event space provides coherent inputs to this array. The units have characteristics of threshold-logic units. There are connections between adjacent array units, indicating that the array units are interconnected.
These systems’ essential parts consist of:
- A collection of processing units that build basic discriminant functions from their input signals after receiving coherent inputs.
- A system that chooses the unit with the highest function value by comparing the discriminant functions.
- A type of local interaction that engages the chosen unit (the “winner”) and its closest neighbors at the same time.
- An adaptive procedure that causes the activated units’ parameters (input weights) to rise in relation to the current input’s discriminant function values.
The Process of Learning
There are two primary stages to the self-organizing process:
Establishment of an activity cluster In the array, an activity cluster develops around the unit where activation peaked. An excitatory area encircled by an inhibitory penumbra and perhaps a weaker excitatory activity beyond that can be used to mimic the process of local lateral contact involved in this phase.
Adaptive change in input weights: The activity cluster’s units’ input weights undergo adaptive modification.
More precisely, each unit i in the array calculates a discriminant function for a given input vector x, which is typically the dot product between its input weights mi and the input vector x (φi = mi · x). The unit k with the highest discriminant function value is chosen by a competitive method. Next, an adaptive method is used to update the weights of the winning unit k and its nearby units.
Mi(t+1) = mi(t) + αx(t) / ||mi(t) + αx(t)|| is the weight update rule for a unit i at time t.
The following are important features of this adaptation:
It is similar to the Perceptron’s teaching rule, except the corrections always follow x’s path (no supervision or decision-making is required). This suggests uncontrolled learning.
A normalization of the weight vectors is performed. It is claimed that normalization preserves “memory resources” at a specific level and increases discrimination selectivity. The main rotation of the process is from mi to x.
Because of the proportionality to the postsynaptic triggering frequency, the correction becomes a function of the distance from the maximum response; nevertheless, this process also impacts the weight vector lengths.
It is thought that this corrective action raises the average differential order throughout the array by enhancing the parallelism of nearby vectors. Changes in individual units lead to a global order since they cannot be independent.
Properties and Capabilities
Topological Mapping: The main function is to create maps in which the spatial arrangement of responsive units in the output array preserves the topological linkages of events in the input space. This extends beyond topographical maps to include maps of patterns associated with arbitrary features.
Unsupervised Learning: Without any intentional supervision, the mapping is created automatically based just on external signal activity.
Feature Extraction: They can learn to become sensitive to various input characteristics, such a signal’s frequency.
Magnification Factor: This model illustrates a feature of biological creatures, which is that the map’s scale or magnification factor is typically not consistent throughout the array. Approximately proportionate to the frequency of input events that have mapped into that area during adaptation, it is site-dependent. This implies that network resources should be used as efficiently as possible based on demand.
The array’s edges may result in “contraction” of the weight vector distribution. The output map expands when the weight vectors compress, which can aid in formatting the map around the edges.
Robustness: Although there are failure conditions, the process is not characterized as being very “brittle” in terms of parameters.
The Perspective of Neural Embodiment
Weight adaptation and activity clustering are two stages that can be put into practice. Lateral interactions in brain structures, possibly involving excitatory and inhibitory regions, are associated with the creation of activity clusters. It is suggested that the adaptive process be implemented by modifications in synaptic efficacies. The required lateral connection may be explained by the branching patterns of many cell types, especially interneurons such as stellate, basket, and chandelier cells. The ideal form of local contact may be tuned by the fraction of a certain cell type.
