How to work with an output of a neural network?

Question

I'm pretty new to neural networks, and I'm trying to make a small network with python. But since I'm new I don't know how to scale and compare the output. So here's my question.

The input I'm giving is a list like this

input = [0,0,1]
desired output = 4

So I do all the neural network coding and after the data passes through the network, the network outputs a number between -1,+1 using a sigmoid function. How can I scale the output to calculate the error with the desired output of 4?

This maybe a stupid question but I'm new to machine learning. Thanks

Answer 1

It's not a stupid question. Want you want to do is normalize your desired output to be in the range that your network outputs(or the inverse and scale the network output up, but the norm is the other way round).

So put your data through a function something like:

def normalize_values(list_of_values, desired_min=-1., desired_max=1.):
    min_val = min(list_of_values)
    max_val = max(list_of_values)
    range = max_val - min_val
    scale = (desired_max-desired_min)/range
    return [((x-min_val)*scale)+desired_min for x in list_of_values)]

Answer 2

In this specific case you probably want a linear output node (ie not sigmoid). Assuming this is a regression problem (see below if it's not). Typically you want a linear output node if it's not supposed to be 'harder' for the NN to predict higher values.

Most machine learning modules allow you to set this.

Sigmoid output nodes are mostly useful for classification problems (because it gets increasingly 'harder' for the neural net to predict values close to 1, which is 100% certain of the class)

If this IS a classification problem, typically the NN performs better with an output node for each class (So use an sigmoid output node for each class, with it's output value corresponding to the chance the NN thinks the sample belongs to that specific class).