XGBoost 训练时间好像太长了

Question

I am trying to train an XGBoost classifier in Python using the xgboost package. I am using the defaults on all the parameters for the classifier and my training set has around 16,000 elements and 180,000 features for each element. I am not using the gpu to train the model, but still, the training process has taken more than five hours and is still going. I have 32GB of RAM and a 6 core Intel I7. I am wondering if this is normal time for training this classifier with the amount of data I have because I have heard of people training the model in a couple of minutes.

Answer 1

If training time is concern then one can switch the tree growing policy tree_method to hist which is histogram based method. With GPU it should be set to gpu_hist . You can find more details about its xgboost implementation here http://arxiv.org/abs/1603.02754

This is the secret sauce which leads to super fast training without much compromise in the solution quality. In fact GPU based training and even lightGBM etc relies on histogram based techniques for faster training and subsequently iterations/experiments which matters a lot in time constrained kaggle type competitions. hist may cut training time to half or less and gpu_hist on gpu may take it to minutes.

PS: I would suggest to reduce the dimensionality of your data (16k X 180k) by removing correlated/rank-correlated features which will further improve not only your training time but also model performance.