Weight Value - Synapse Weight - A value used to modulate, or gate a single input signal (called an axon level) stimulating a neuron during signal propagation phase. Expressed in floating-point arithmetic, the value of the input signal is simply multiplied by the weight value to produce a result. The result of the multiplication is then summed by the neuron. During training or learning phase, weight values are changed in order to bring each neuron's output response in line with a desired response for a given set of inputs.

In conventional ANN models, a synapse's weight is a single, floating point, number that represents the connection-strength, and the type (inhibitory or excitatory) of a given connection. A negative value represents an inhibitory connection, and a positive value represents an excitatory connection.


. . . . . . .
Weight Adjustments in Netlab

In Netlab weight-values (or at least their signs) are abstracted. It bases its weight increase and decrease operations on connection strengths. The absolute value of the weight represents the strength of the connection, so a call for an increase may increase or decrease a conceptual "signed" weight-value depending on its sign. While increasing and decreasing an excitatory (positive value) weight will have the expected effect. An increase to an inhibitory weight-value (traditionally, a negative signed value) will increase the absolute value, which would be the same as reducing the value of a signed weight-value. Nelab also allows weights to be set to respond in the traditional way to increase and decrease signals if the designer chooses. This is discussed in the section below title "Netlab's Compatibility Mode."

Netlab also allows a designer to choose how (or if) a given synapse will cross-over from one type (say, Excitatory), to the other type (Inhibitory). This can be set to remain only the initially designed type, or synapses can be set to be allowed to change between types. Finally, they can be set to change, or not change, based on run-time signaling.

In the case where they are set to change freely between types (the default), the algorithm is this:

• Crossing through (or from) zero
  If a weight-change is called for that takes it through zero (either because it is greater than the value's distance from zero, or because the weight is at zero when the delta-value is applied) one of two things will happen:
  • A negative (decrease) adjustment is called for
    1. Subtract the current value from the adjustment-amount.
    2. Set the synapse-type to Inhibitory and the absolute value equal to the remaining adjustment amount.
  • A positive (increase) adjustment is called for
    1. Subtract the current absolute weight-value from the increase-amount.
    2. Set the synapse-type to Excitatory, and its absolute value equal to the remaining increase-amount.
  
  

Netlab's default behavior is to allow weights to change freely between synapse-types (excitatory or inhibitory) based on the above algorithm.



. . . . . . .
Netlab's Compatibility Mode

ANN models that use floating point signed-value weights in the conventional fashion are math-centric. That is, they typically are concerned only with the signed numeric weight-value, rather than with the connection-strength represented by its absolute value. In this case, for example, increasing the weight value will make it more positive, regardless of whether it is representing an excitatory or inhibitory connection. While this increases the connection-strength of excitatory (positive) weight-values, it reduces the connection-strength of inhibitory (negative) weight values.

Netlab's default behavior is to operate directly on connection-strength representations, regardless of how they are implemented internally. Netlab's Noodle™ will facilitate the conventional practice, however, by allowing it to be specified at the weight-layer learning method.

The table below shows how Netlab facilitates compatibility with existing practices. The table documents how the translation is carried out between the traditional math-centric convention, and Netlab's connection-strength-centric convention.

Connection-Type—> v—Operation	Excitatory	Inhibitory
Increase	Increase Connection Strength	Decrease Connection Strength
Decrease	Decrease Connection Strength	Increase Connection Strength

Translations performed when conventional practice is specified for a connection.


Because Netlab will normally default to allow synapses to change between types based on the algorithm presented in the last section, it will already behave in the traditional way. Of course, a designer is free to use traditional-style compatibility for weight adjustments, while applying constraints as to how and if synapses are allowed to change their type. That is, in the case of a conceptual weight value, whether it is permitted to cross through the zero-mark.