In neural networks, information is represented conceptually in the strengths of the connections between neurons, or between outside signal sources and neurons. Connection strengths are encoded in the values of connection weights at the inputs of neurons. Input signals are modulated by the connection weight values at the inputs, which, in turn determine how much of the input signal is conveyed by the connection synapse.


. . . . . . . . . . .
Connection-Strength vs. Weight Value

Conventionally, Weight values are numbers representing the connection-strength and type (inhibitory or excitatory) of a connection. The connection-type (inhibitory or excitatory) can be accommodated in a variety of ways, but usually the type is represented by the sign of the weight-value. If the weight-value is negative, the weight represents an inhibitory connection and if the weight-value is positive it represents an excitatory connection.

There is some confusion associated with using this signed representation to represent type:

• inhibitory connection-strengths are enhanced (made stronger) by being decreased (by being made more negative). Conversely, they are reduced by being increased (by being made less negative).

• Excitatory connection-strengths, on the other hand, are enhanced by being increased (made more positive), and reduced by being decreased (made less positive).
  

Stated more generally, a connection-strength's relationship to a conventional signed weight-value is that the connection-strength is reduced when the value moves closer to zero, and the connection-strength is enhanced when the weight-value is moved farther away from zero. Regardless of if it is negative (inhibitory), or positive (excitatory).

One more way to say it: The strength of the connection is represented in the magnitude of the absolute value of the weight, regardless of its type (inhibitory or excitatory)



. . . . . . .
Alternative Representations

There are many ways to represent connection-strengths in neural networks. Any mechanism that can be altered in a way that can mimic the effect of changing strengths of a connection between two entities can be used to represent connection-strengths. A variable resistor is probably the most obvious example of this. These days, in fact, a lot of excitement is being generated around the possibility of using memristors as weights in neural network circuits.

That said, it is important to understand that any mechanism that serves to modulate the strength of a connection between two things can be thought of, in a general sense, as a provider of connection-strengths. To give an extreme example, a gate between two fenced areas can be left half open, thereby modulating the connection strength between the two areas by half. This reduces the flow of live-stock between them.