简体   繁体   中英

How can i save a machine learning model that takes less memory

I am training a RandomForest Classifier with somewhat large dataset of around 580mb and it is taking more than 30 min to fit. Now when I try to save the model using joblib then the saved model takes around 11.1gb of space. Is it normal or I can save the model more efficiently in terms of space consumed as I am thinking of deploying the model.

Is it worth using a model that takes so much space because I have a decision tree model on same data that takes 278mb space and it's accuracy is just 2% lower(91%)

my notebook

This is model saving code

from sklearn.externals import joblib  
# Save the model as a pickle in a file 
joblib.dump(Random_classifier, '/content/drive/My Drive/Random_classifier.pkl') 

I am a newbie so don't vote to close the question just leave a comment. I am willing to edit the question asap.

Random Forest classification method is way expensive in memory. try to lower your decision trees number, might reduce some memory. It seems your dataset is also very big so I think it seems legit your weights size. Also I know there is pickle way to save weights, I would recommend to check it out too.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM