神经网络最小化 Function

Question

I created a simple neural.network; in order to actually train it, I would need to know in which direction the weights and biases need to be tweaked. I've read some articles on the topic, but I'm not exactly great at math and the only thing I understood was that the cost functions (which I managed to get working) need to be minimized. It would be great if someone could at least tell me in theory how this works. If required, I could also post more of the code. The minimize function should in the end replace evolve() :

import java.util.Random;

public class Neuron {
Neuron[] input;
float[] weight;
float bias;
Float value = null;
public Neuron(Neuron[] input) {
    this.input = input;
    weight = new float[input.length];
    setRandom();
}
public void setValue(float val) {
    this.value = val;
}
public float getValue() {
    if(this.value == null) {
        return calculate();
    }
    else {
        return this.value;
    }
}
private float calculate() {
    float res = 0;
    for(int i = 0; i < input.length; i++) {
        res += input[i].getValue() * weight[i];
    }
    res -= bias;
    return sigmoid(res);
}
private void setRandom() {
    Random rand = new Random();
    float max = 0;
    for(int i = 0; i < weight.length; i++) {
        weight[i] = rand.nextFloat();
        max += weight[i];
    }
    this.bias = max * 0.8f - rand.nextFloat();
}
public void evolve() {
    Random rand = new Random();
    for(int i = 0; i < weight.length; i++) {
        weight[i] += rand.nextFloat() - 0.5f;
    }
    this.bias += rand.nextFloat() - 0.5f;
}

public static float sigmoid(float x) {
    return (float)(1/( 1 + Math.pow(Math.E,(-1*(double)x))));
  }
}

Answer 1

Cost function is basically a function of the difference between the real datapoints and your predictions (ie it's your penalty). Say for argument's sake, your neural.network is f(x) = 2x + 1 . Now, say your observed real datapoint is x = 1 , y = 4 . Therefore your prediction ( f(1) ) is 3 .

If your cost function is the absolute difference between actual observed value and prediction ie |f(x) - y| the value of your cost function is 1 (for x = 1 ) and you would need to minimize this cost function. However, if your cost function is 100 - |f(x) - y| you would want to maximize it. In this cost function your maximum reward is 100 .

So your weights and bias need to move in the direction that would get you closer to minimizing your penalty and maximizing your reward. The closer your prediction is to the observed dataset value, the higher the reward and smaller the penalty.

Notes :

1. This is a gross oversimplification of the math involved but it should help you get started. Also read about overfitting in machine learning.

2. For understanding machine learning theory Cross Validated would be better forum.