How can I save the state of running python programs to resume later?

Question

I am developing the machine learning analysis program which has to process the 27GB of text files in linux. Although my production system won't be rebooted very often but I need to test that in my home computer or development environment.

Now I have power failure very often so I can hardly run it continuously for 3 weeks.

My programs reads the files, applies some parsing, saves the filtered data in new files in dictionary, then I apply the algorithm on those files then saves result in mysqlDB.

I am not able to find how can I save the algorithm state.

Answer 1

我关于算法状态的所有内容都保存在一个类中，您可以序列化该类并将其保存到磁盘： http : //docs.python.org/2/library/pickle.html

Answer 2

Since the entire algorithm state can be saved in a class, you might want to use pickle (as mentioned above), but pickle comes with it's own overloads and risks .

For better ways to do the same, you might want to check out this article , which explains why you should use the camel library instead of pickle .