Weight Value -
Synapse Weight - A value used to modulate, or
gate a single input signal (called an
axon level) stimulating a
neuron during signal propagation phase. Expressed in floating-point arithmetic, the value of the input signal is simply multiplied by the
weight value to produce a result. The result of the multiplication is then summed by the neuron. During training or
learning phase,
weight values are changed in order to bring each neuron's output response in line with a desired response for a given set of inputs.
In conventional ANN models, a
synapse's
weight is a single, floating point, number that represents the
connection-strength, and the type (
inhibitory or
excitatory) of a given connection. A negative value represents an inhibitory connection, and a positive value represents an excitatory connection. The absolute value of the
weight represents the strength of the connection.
. . . . . . .
Netlab's Compatibility Mode
ANN models that use floating point signed-value weights in the conventional fashion are math-centric. That is, they typically are concerned only with the signed numeric weight-value, rather than with the connection-strength represented by its absolute value. In this case, for example, increasing the weight
value will make it more positive, regardless of whether it is representing an excitatory or inhibitory connection. While this increases the
connection-strength of
excitatory (positive) weight-values, it
reduces the connection-strength of inhibitory (negative) weight values.
Netlab's default behavior is to operate directly on
connection-strength representations, regardless of how they are implemented internally. Netlab's Noodle™ will facilitate the conventional practice, however, by allowing it to be specified at the weight-layer learning method.
The table below shows how Netlab facilitates compatibility with existing practices. The table documents how the translation is carried out between the traditional math-centric convention, and Netlab's connection-strength-centric convention.
Connection-Type->
v--Operation
|
Excitatory
|
Inhibitory
|
Increase
|
Increase Connection Strength
|
Decrease Connection Strength
|
Decrease
|
Decrease Connection Strength
|
Increase Connection Strength
|
Translations performed when conventional practice is specified for a connection.