How can I find the reason that Python script is killed?

Question

I have about 1 million files (which are output simulation outputs). I want to store a specific information of them within one file. I have a for loop which goes to 1M. I put a counter to track the state of the for loop. It will be killed some where between 875000 and 900000. I though that it may be a space problem. When I run df -h or df / , I have about 68G available. What are other possible reasons that a Python script may be killed? How can I explore it more?

Answer 1

Usually, you get killed message when the program runs out of RAM (as opposed to hard disk which you have in plenty). You should keep a watch on main memory. Run top and have a look at the memory being taken by your program, or alternatively use a tool like guppy ( https://pypi.python.org/pypi/guppy/ ) to track memory utilization programmatically.

I would hazard a guess that you are creating some big in memory data structure while processing files, perhaps not de-allocating them as you iterate through the files.

Answer 2

On a Linux system, check the output of dmesg . If the process is getting killed by the kernel, it will have a explanation there. Most probable reason: out of memory, or out of file descriptors.

Answer 3

Snippet of code will help. However, I presume, you're loading all the files in memory in one go and since files are huge, that might have bloated RAM completely, and thus making script to die. If your use case is to get particular line/text from every file, I would recommend to use re modules for pattern and read files accordingly.

Please refer syslog . You can get syslog in /var/log/ in Ubuntu. syslog will give you hints of possible reasons of the script failure