Encog neural network training java too slow

Question

I am training a neural network to classify images and it takes too long to finish one iteration... about five minutes and it is still not done. I am using Encog 3.1. Is there something wrong with my code?

BasicNetwork network = new BasicNetwork();
        network.addLayer(new BasicLayer(null,true,5625));
        network.addLayer(new BasicLayer(new ActivationSigmoid(),true,(intIdealCount+5625)/2));
        network.addLayer(new BasicLayer(new ActivationSigmoid(),true,intIdealCount));
        network.getStructure().finalizeStructure();

here is my training codes:

final ResilientPropagation train = new ResilientPropagation(network, trainingSet);

        int epoch = 1;

        do {
            train.iteration();
            System.out.println("Epoch #" + epoch + " Error:" + train.getError());
            epoch++;
        } while(train.getError() > 0.01);

Any response will be appreciated. Thank you.

Answer 1

Your code seems fine, but training can get arbitrary long depending on your data. From the size of your network one can deduce, that you are working with images - now if you have lots of them - even the most efficient implementation will take forever. Encog is quite good piece of code - it by default works on all avaliable cores, but FANN seems to be the fastest library for ANN for now.

You have ~5000 input neurons, assuming that you have ~10 output neurons, you have ~2500 hidden ones. So your network has (5000+1)*2500 + (2500+1)*10 weights (about 12,500,000). Now, assuming that you have N images in your training set - one epoch requires computation (and update) of 12,500,000 * N values. So even if you have just ~200 images it is 2,500,000,000 updates to compute.

There are at least three possible ways:

• Try the FANN library, which is one of the most efficient ones
• Reduce dimensionality of your images using for example PCA (and as a result - reduce the size of the network)
• Are you sure that you need 2500 hidden nodes? It is quite a lot