有没有办法在 Python 中并行化集成学习循环？

Question

I want to train multiple LightGBM models simultaneously.

Right now, I'm training them sequentially like below:

for m in range(ensemble_n):
   params = {'seed':m}
   model = lgb.train(params, lgbtrain)
   prediction=model.predict(test_df.drop([target], axis=1))
   test_predictions[:, m] = prediction

Is there a way for me to parallelize the loop above?

Answer 1

Training multiple versions of a model in parallel comes at a cost that you need to have multiple versions of the data loaded into memory, which can get difficult if you have a sizeable dataset.

At the same time, if you're using a scikit-learn API of LGBM, you can utilise the parameter n_jobs=-1 which will parallelize calculations of a single model over all available cores. This would be a more efficient use of resources, because regardless you'll have to chose either to train multiple models in parallel or to train a single model in parallel, but not both.