JavaScript: Simple Perceptron fails to train

Question

I tried to make simple perceptron for predicting true or false, where 1 means true and false means -1. but my perceptron fails to train itself and instead gives random values.

this is my class perceptron:

 function f(x){ //number is in negative it returns -1 if(x<0){ return -1 //number is in positive it returns 1 }else{ return 1 } } class Perceptron{ constructor(){ //two weights this.weights = new Array(2) //intitializing two random weights for(let i = 0; i<this.weights.length; i++){ this.weights[i] = Math.random()*2-1 } } //prediction guess(input){ let sum = 0; for(let i = 0; i<input.length; i++){ sum += this.weights[i]*input[i]//sum of all product of inputs and weights } return f(sum)//returns -1 or 1 } //training data train(inputs, target){ this.lr = 0.1//learning rate let guess = this.guess(inputs)//answer comes either 1 or -1 let err = target - guess//calc error for(let i = 0; i<this.weights.length; i++){ this.weights[i] += err * inputs[i] * this.lr// re adjust the weights } } } export default Perceptron;

This is my main js:

 import Perceptron from "./perceptron.js" const brain = new Perceptron; let data = [{ inputx: 1, inputy: 1, target: 1 }, { inputx: 0, inputy: 1, target: -1 }, { inputx: 1, inputy: 0, target: -1 }, { inputx: 0, inputy: 0, target: -1 } ] for(let i = 0; i<data.length; i++){ let inputs = [data[i].inputx, data[i].inputy] brain.train(inputs, data[i].target) let guess = brain.guess([1, 0]) console.log(guess) }

the perceptron does not predict exact data, instead gives random answers. What am i doing wrong here

Answer 1

I put your code in a snippet, so you can run it here.

 function f(x) { //number is in negative it returns -1 if (x < 0) { return -1 //number is in positive it returns 1 } else { return 1 } } class Perceptron { constructor() { //two weights this.weights = new Array(2) //intitializing two random weights for (let i = 0; i < this.weights.length; i++) { this.weights[i] = Math.random() * 2 - 1 } } //prediction guess(input) { let sum = 0; for (let i = 0; i < input.length; i++) { sum += this.weights[i] * input[i] //sum of all product of inputs and weights } return f(sum) //returns -1 or 1 } //training data train(inputs, target) { this.lr = 0.1 //learning rate const guess = this.guess(inputs) //answer comes either 1 or -1 const err = target - guess //calc error for (let i = 0; i < this.weights.length; i++) { this.weights[i] += err * inputs[i] * this.lr // re adjust the weights } } } const brain = new Perceptron; let data = [{ inputx: 1, inputy: 1, target: 1 }, { inputx: 0, inputy: 1, target: -1 }, { inputx: 1, inputy: 0, target: -1 }, { inputx: 0, inputy: 0, target: -1 } ] for (let i = 0; i < data.length; i++) { let inputs = [data[i].inputx, data[i].inputy] brain.train(inputs, data[i].target) let guess = brain.guess([1, 0]) console.log(guess) }

From the code what I see:

1. Perceptron initialised with random weights - OK
2. Perceptron fed with data - OK

If you analyse the guessing function, then you'll see some problems:

• guess[1, 1] : the weights are added up. Likely that their sum is 0+, so the guess will yield a correct answer most of the time
• guess[0, 1] and guess[1, 0] : only the weight that's paired with the 1 is taken into account (the other is multiplied by 0, then added to the sum); there's a slightly larger chance to yield a 1 than a -1 (that means false guess), but mostly random
• guess[0, 0] is going to be always false , as it's internal sum (in the guess function) is going to be always 0, therefore yield a 1 from f(x) -> target is -1 (false result)

The Perceptron works - as it modifies the weights during "learning", but is not going to yield significantly different answers than a random guess would.

MODIFIED GUESS FUNCTION

 function f(x) { //number is in negative it returns -1 if (x < 0) { return -1 //number is in positive it returns 1 } else { return 1 } } class Perceptron { constructor() { //two weights this.weights = new Array(2) //intitializing two random weights for (let i = 0; i < this.weights.length; i++) { this.weights[i] = Math.random() * 2 - 1 } } //prediction guess(input) { let sum = 0; for (let i = 0; i < input.length; i++) { // ----------------------- // changed the * to + here // ----------------------- sum += this.weights[i] + input[i] //sum of all product of inputs and weights } return f(sum) //returns -1 or 1 } //training data train(inputs, target) { this.lr = 0.1 //learning rate const guess = this.guess(inputs) //answer comes either 1 or -1 const err = target - guess //calc error for (let i = 0; i < this.weights.length; i++) { this.weights[i] += err * inputs[i] * this.lr // re adjust the weights } } } const brain = new Perceptron; let data = [{ inputx: 1, inputy: 1, target: 1 }, { inputx: 0, inputy: 1, target: -1 }, { inputx: 1, inputy: 0, target: -1 }, { inputx: 0, inputy: 0, target: -1 } ] for (let j = 0; j < 1; j++) { for (let i = 0; i < data.length; i++) { let inputs = [data[i].inputx, data[i].inputy] brain.train(inputs, data[i].target) } } let guess = 0 console.log('After 1 round of training:') guess = brain.guess([1, 1]) console.log(guess) guess = brain.guess([0, 1]) console.log(guess) guess = brain.guess([1, 0]) console.log(guess) guess = brain.guess([0, 0]) console.log(guess) console.log('Weights:', brain.weights) for (let j = 0; j < 999; j++) { for (let i = 0; i < data.length; i++) { let inputs = [data[i].inputx, data[i].inputy] brain.train(inputs, data[i].target) } } console.log('After 1000 round of training:') guess = brain.guess([1, 1]) console.log(guess) guess = brain.guess([0, 1]) console.log(guess) guess = brain.guess([1, 0]) console.log(guess) guess = brain.guess([0, 0]) console.log(guess) console.log('Weights:', brain.weights)

I changed the guess function

// from this:
sum += this.weights[i] * input[i]
// to this
sum += this.weights[i] + input[i]

and added some more rounds of training. I think the results speak for themselves (run it a couple of times to see the difference in 1 round of training vs. 1000 rounds of training).