Hidden Layer in a Neural Network

Question

So I'm currently completing my Udacity Nanodegree and there is this one point that i'm not able to understand. In the input layer we're taking 784 because the pixel of MNIST dataset is 28x28, but i don't understand why are they taking 256 hidden layer. Where did this 256 number came from?

Answer 1

The 256 hidden layer represent the 0 to 255 value given to each pixel to represent saturation of color for that pixel.

Answer 2

The number of units in hidden layers is an hyperparameter of the.network, so just like any other hyperparameter (like the learning rate, regularization factor and so on) it is initially chosen arbitrarily, then its value is "tuned" by assessing the model's performances on the validation set.

Sometimes though, especially in small and contrived examples like MNIST, some hyperparameters are fixed once and for all and never tuned, given how simple the task at hand is. So yeah, chances are that the number 256 was chosen arbitrarily.