PyCharm IDE 调试时速度很慢 Python 读取小 pickle 文件

Question

Debug takes sometimes 10 seconds. The problem starts after reading 30mega + 5 mega pickles, with with pdb debugging is fine. Even on lines of 'print'

any suggestions?

if __name__ == "__main__":
    print 'loading files...',
    with open(fname1, 'rb') as handle: items = pickle.load(handle)
    with open(fname2, 'rb') as handle: sentences_by_id= pickle.load(handle)
    print ' done!'

Answer 1

I had a similar problem with a dict of lists of ndarrays. when the list was too long, the pycharm debugger would freeze, top would show cpu for the pycharm at 102% ( I guess several cores were active) and it would take a very long time to unfreeze. Running without breakpoints was fine. My solution was to convert all my lists on ndarrays to ndarrays, eg

<pseudo python>
for k in dict_of_lists_of_ndarrays:
     dict_of_lists_of_ndarrays[k]=np.array(dict_of_lists_of_ndarrays[k])
</pseudo python>

now things are much improved. I have no idea what the issue was

Answer 2

I found this thread on JetBrains support, which is suggesting to turn off "Show return values" option (you can find it in the settings of the Debug tool bar, the gear icon). It has solved it for me.

Answer 3

I did some analysis on this, also changed my IDE to Elipse using PyDev plugin.

Still debugging python program was way too slower than the actual directly executing python program.

How much slower? It took my computationally intensive program more than 17 minutes and still no solution where when run directly, the same program returned output within 2 minutes.

I guess the problem is with Python Debugger, not with either of IDE's Eclipse or Jetbrains Pycharm. I can vouch for both as I have worked on both for many years.

It's worth checking if there's any explicit debugger speed setting option available for Python Debugger.

Answer 4

What help for me was to change the "variable loading policy" to "on demand". This is done by clicking the small cog wheel in the debug menu. For a picture check the link:

https://intellij-support.jetbrains.com/hc/en-us/community/posts/206601625-pycharm-debugger-slow-due-to-collecting-data-

Answer 5

this happened to me (PyCharm 2019.3) and it was caused by invalid breakpoints that had been placed on files subsequently deleted, edited etc. Delete all breakpoints and make sure you haven't got a breakpoint on Exception "raise". Once I did that PyCharm debugging went back to normal speed

Answer 6

quite in delay, however i had similar issues that couldn't be solved for the life of me for more than a week. Tried all the above, and much more. Eventually, for me one of the two following solutions worked (found them in the inte.net, unfortunately can't find where, for proper credit):

1. Python Debugger -> Enable Gevent.
2. Add at the beginning of the run np.set_printoptions(threshold=5).

Can't tell why it helped - I didn't change anything in my code, including usage of gevents, when the issue started. But, it seems to be of no harm; especially option (2)...

#voodoosolutions