A neural network is the way that engineers and scientists look at an organic brain and simulate it. The question is how should it be organised and trained?
In the 1950's there were single layer networks. Their limitations meant they were not good at learning.
In the 1970's it got better when backpropagation was invented. Results were still too slow. Finally in the 1990's pre-training came in. The lower layers of the network absorb some of the input so the later layers can do the work more easily. The two branches of this science are:
- Auto-encoders reproduce their inputs as outputs. It remembers general features e.g. the shape of a nose and mouth for face recognition. This is achieved by giving the lower layers less memory than they need to remember the exact features of the system. Next expose the auto-encoder to a large number of samples of the system e.g. thousands of pictures of faces. The general features are trained into all layers of the system. When this is done backpropagation is used to fine tune the recognition settings.
- Boltzmann machines are used for noisy/statistical data. This reproduces a statistical distribution of input data and is used in place of auto-encoders. Instead of general representational features, general statistical features are trained into the layers instead e.g. Voice frequencies, hospital data on cancer patients.
It could be also that in the future if there is more computer processing power available that pre-training will not be needed to help neural networks learn faster.
Author: Seamus Taylor
Post #: 5
No comments:
Post a Comment