学习人工神经网络特性？

Question

I have a data-set with around 50,000 properties per item. (mostly values between 0 an 1, no discrete values at all)

The properties are not labelled, and assumed to have no relation with each other. + I know in advance that most properties are useless. (99% of them)

My task is to use as few properties in a neural network, such that it would know how to differentiate between 5 item types.

In theory, I could just through all 50K properties into the ANN, and hope for the best, but it would take a huge amount of time to train. + gigabytes of RAM, and I am not sure my server won't crash.

Is there a model that measures the level of classification a parameter has?

If not, would the following be a good idea?

• Go over all of my 50K parameters, and train 50K ANNS, with <1, parameter>
• Get the maximum accuracy ANN, and start again, with 3 inputs: <1, previous-property, property>, and so on, until I get to an accuracy of 95% and then stop

I see no reason it won't work, but training at least 10*50,000 ANNs is not ideal as well.

EDIT:

I have 12 examples per category. overall 60 items. (I am aware it is tiny, but I can't get more.)

Answer 1

Feature Selection

I would shy away from a neural network to solve this problem. If you are tied to the neural network idea, then it would be possible to plug in your 50000 x 60 data matrix to the network as this shouldn't take very much ram at all. If you use an L1 regularizer, then analyze the weights of the network afterwards for all 0 entries, you can determine which features were not useful.

There are numerous other feature selection approaches as well. For instance the LASSO algorithm attempts to solve this problem in a very similar way to the above neural network approach.

Another well known algorithm is forward selection regression, where you perform a regression using only one property at a time. You then pick the attribute that best separates the classes, fix that property then select again using two properties at a time (the best property from the last sweep, and every other property one at a time). You repeat this process until adding another property gives no better class separation. I would not be concerned with the time it takes to train this model if most features truly are useless. Using linear regression (as it has a closed form solution) should take almost no time at all on a dataset of this size.

Feature Extraction

A much more principled approach would be some form of principle components analysis (PCA). This would show you how many collinear properties your dataset has, and would extract a small number of new properties to describe your data.