A neural network is an artificial intelligence model used frequently in learning applications. The first step to understanding how neural network simulators, used to test configurations of neural network models and algorithms, is to experiment with programming and training a neuron.
Neural networks are comprised of neurons connected together in a variety of configurations, and this article deals with a single neuron specifically. This article examines how to:
- Program Neurons
- Train Neurons
- Recognize Patterns
The result is that the reader should be able to understand and use a neural network simulator, create their own neural networks, and program neurons into applications.
The Neuron
A neuron is a simple mechanism that is derived from existing understanding of animal's brains, including the human brain. Each neuron is a fuzzy decision unit that receives one or more inputs and generates an output if certain conditions are met.
Those conditions include a summing of the weighted input values followed by the application of a function to the result which yields a value that is then compared to a threshold value. If the calculated importance of the signal passes the threshold, then the neuron generates an output (usually a high value, equivalent to the Boolean TRUE).
Neuron Programming
In practice, when implementing a neuron, it is best to start with a very simple premise - taking multiple weighted inputs and generating a single output. A weighted input is simply a value to which a multiplier is attached that affects it's perceived value. Some inputs will have more importance attached to them than others, so they are given a larger weight.
These weighted input values are then added together, which is used as the output. For simplicity's sake, this value can be interpreted directly, rather than, at this point, applying a threshold or modification function. Those come later.
A neuron can have multiple inputs, and it makes sense to use a system of weights that add up to 100%. It is the modification of the weights which allow the neuron to be trained.
In C, a neuron might look like the following:
{
float fInputWeights[NUM_INPUTS];
float fInputValues[NUM_INPUTS];
float fOutput;
};
It is assumed that NUM_INPUTS has been set to something sensible.
Training & Recognition
Neurons can be used to identify whether an image is close to a predefined pattern. Usually a value close to 1 (say 0.9) is used to indicate that this is the case, and a value close to 0 (say 0.1) if it is not.
A neuron can then be established with as many inputs as required. For an 8x8 grid, that would be 64 inputs.
The starting point is to randomize the weights. Then, the pattern is set on the inputs, and each input is multiplied by it's weight. For each part of the pattern that is shaded, the input is set to 1, for the rest, the input is set to 0.
Having summed the inputs, there are then several options:
- Take the average (divide by 64)
- Apply a sigma function (mimics human learning)
To train the network, the output is compared with a value that indicates success - 0.9 in this case - and the deviation from that value measured. With this in hand, the weights are then altered - setting the pattern input to 1 by a proportion (say 1/100th) of that difference. This strengthens the importance of those inputs.
If this is done a number of times, the confidence with which the neuron will be able to match the pattern will increase. Recognition then just becomes a case of showing the neuron the pattern, and then measuring the confidence. With enough neurons any pattern can be matched in this way.
Of course, to get higher recognition rates, layers of neurons can be coupled together in a neural network. The above explanation, however, should be enough for the reader to start their own experiments with neural network algorithms and a neural network simulator.
