How To Evaluate Predictive Neural Network in Encog

Question

We're a creating a neural network for predicting typhoon occurrence using several typhoon parameters as input. So far, we have been able to generate data and train the neural network using Encog 3.2 . Right now, we need to evaluate the results of the training.

We're using the ForestCover project (in Encog 3.2 examples) as reference, however the said project's Evaluation code is for a classification neural network. Thus, we can't evaluate our neural network following said project's code.

We also checked the PredictMarket project (in Encog 3.2 examples) since it is a predictive neural network. But we're having a difficulty in using the MLData.

MLData output = network.compute(inputData);

We want to extract the contents of output and compare it to the contents of the evaluation.csv for neural-network evaluation.

Is there a way we can extract/convert the output variable into a normalized value which we can then compare to the normalized evaluation.csv?

or

Is there a way we can modify the ForestCover Evaluate.java file to be able to evaluate a predictive neural network?

Thank you.

Answer 1

Here is a C# example (Java should be similar) which writes out a .csv file (TestResultsFile) with both the de-normalized expected and actual results so you can compare them with an Excel graph.

var evaluationSet = (BasicMLDataSet)EncogUtility.LoadCSV2Memory(Config.EvaluationNormalizedFile.ToString(),
    network.InputCount, network.OutputCount, true, CSVFormat.English,
        false);

var analyst = new EncogAnalyst();
analyst.Load(Config.NormalizationAnalystFile);

// Change this to whatever your output field index is
int outputFieldIndex = 29;

using (var resultsFile = new System.IO.StreamWriter(Config.TestResultsFile.ToString()))
{
    foreach (var item in evaluationSet)
    {
        var normalizedActualOuput = (BasicMLData)network.Compute(item.Input);
        var actualOutput = analyst.Script.Normalize.NormalizedFields[outputFieldIndex].DeNormalize(normalizedActualOuput.Data[0]);

        var idealOutput = analyst.Script.Normalize.NormalizedFields[outputFieldIndex].DeNormalize(item.Ideal[0]);


        var resultLine = String.Format("{0},{1}", idealOutput, actualOutput);
        resultsFile.WriteLine(resultLine);

    }
}

Much of this came from ideas from Abishek Kumar's Pluralsight Course

If you really want to compare Normalized values, just remove the two calls to "Denormalize".