气流：存储机器学习模型

Question

I'd make pipeline of machine learning on Airflow.
Example)

result = model.fit()

But DAG file:( sample.py ) is refreshed on each times.

So I can't store trained model.

How should I treat trained model ?

Or should I store the trained model data in external?
Then take the trained model in use?

Answer 1

Store the model in pickle file and retrieve the model from same file whenever you want.

import pickle 
result = model.fit()
# save the model to disk
filename = 'my_model.sav'
pickle.dump(model, open(filename, 'wb'))

# load the model from file for later use
reload_model = pickle.load(open(filename, 'rb'))
result_final = reload_model.score(X_test, y_test)
print result_final