SciKit One-class SVM classifier training time increases exponentially with size of training data

Question

I am using the Python SciKit OneClass SVM classifier to detect outliers in lines of text. The text is converted to numerical features first using bag of words and TF-IDF.

When I train (fit) the classifier running on my computer, the time seems to increase exponentially with the number of items in the training set:

Number of items in training data and training time taken: 10K: 1 sec, 15K: 2 sec, 20K: 8 sec, 25k: 12 sec, 30K: 16 sec, 45K: 44 sec.

Is there anything I can do to reduce the time taken for training, and avoid that this will become too long when training data size increases to a couple of hundred thousand items ?

Answer 1

Well scikit's SVM is a high-level implementation so there is only so much you can do, and in terms of speed, from their website, "SVMs do not directly provide probability estimates, these are calculated using an expensive five-fold cross-validation."

You can increase your kernel size parameter based on your available RAM, but this increase does not help much.

You can try changing your kernel, though your model might be incorrect.

Here is some advice from http://scikit-learn.org/stable/modules/svm.html#tips-on-practical-use : Scale your data.

Otherwise, don't use scikit and implement it yourself using neural nets.

Answer 2

Hope I'm not too late. OCSVM, and SVM, is resource hungry, and the length/time relationship is quadratic (the numbers you show follow this). If you can, see if Isolation Forest or Local Outlier Factor work for you, but if you're considering applying on a lengthier dataset I would suggest creating a manual AD model that closely resembles the context of these off-the-shelf solutions. By doing this then you should be able to work either in parallel or with threads.

Answer 3

For anyone coming here from Google, sklearn has implemented SGDOneClassSVM , which "has a linear complexity in the number of training samples". It should be faster for large datasets.