SVM-是否存在可以指示最佳参数（例如C，伽玛）的数据属性

Question

It seems pretty standard to use cross validation to determine the best parameters. Of course, this is usually a time-consuming process. Are there any shortcuts? Are there other, faster, forms of exploratory analysis that can provide a hint as to which values will be best?

For example, at my current understanding of machine learning and SVM, I might do something like perform an initial grid search in the range of [10e-5, 10e5] at exponents of 10 for C, and then fine tune from there. But is there a way I could quickly estimate that the best C is somewhere between 10e3 and 10e5, and then perform more specific searches?

This question probably applies to most ML techniques, but I happen to be working with SVM right now.

Answer 1

Yes, this is an area of active research! There has been a lot of work in different approaches to hyper-parameter tuning besides the standard grid search we all know and (maybe?) love.

The area most similar to what you are describing are various bayesian / gaussian process approaches to the problem. This github repo has an implementation and some informative pictures on how it works https://github.com/fmfn/BayesianOptimization . This approach works by treating the parameter optimization problem as another machine learning problem, where we have features for every hyperparameter, and try to predict the performance of various parameter combinations.

That is a high level description of the process, you can read the linked papers/notebooks in the repo for more details.