Python: print() always writes to .ipynb making file too large

Question

To cut to the chase, I'd like to know how to make print statements display in ipython notebooks while simultaneously preventing those statements from saving to the .ipynb file. The purpose being to make a progress bar which doesn't make the file size ridiculously large.

The background to this is that I've been writing a bit of python code which makes a bunch of png files so that I can eventually compile them into a GIF. While I was doing this I thought I'd be clever and print the progress of the task as it went using print() from __future__ with carriage returns. Unfortunately though I've had two problem with my code, both of which I imagine are related to my implementation of this progress message.

The first problem is with github's limit on file size:

When I first tried to upload my file to github it prevented me from doing so because it exceeded their 100 MB limit. After investigating my .ipynb file I found that there was an obscene number of print statements which were being saved there. Initially I'd thought that including '\\r' to do carriage returns would prevent this, but clearly that's not the case.

The second problem is probably related to this:

Typically I don't have a problem creating the first few GIFs especially if I don't include that many frames, however beyond that my python notebook crashes. If this were a typical memory problem I'd imagine that it would just throw an error at me, but it doesn't, and instead promptly dies on me.

Here's a sample of the sort of stuff that's bloating the .ipynb file:

{
   "output_type": "stream",
   "stream": "stdout",
   "text": [
    "\r",
    "frame 1 -- 0.480% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.500% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.520% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.540% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.560% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.580% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.600% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.620% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.640% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.660% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.680% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.700% complete -- U_avg = -7.200000\r",
    "frame 1 -- 0.720% complete -- U_avg = -7.200000\r", ......

I've looked into how other people have implemented progress bars, however they don't appear to do anything special which would actually prevent the problem I'm having. If it comes down to it, I wouldn't mind importing something which would solve this problem in a black box, but at the same time if I run into this issue in a different context it would be useful to know how to reduce .ipynb file sizes by cutting out saved print statements.

Thanks!

Answer 1

I can think of two possible solutions:

1. Add a conditional so that you only print when significant progress has been made? ie only print whole percentage points.

2. Use Javascript to display and update your progress bar. The Javascript need to periodically call the ipython kernel to get progress update. Example: https://jakevdp.github.io/blog/2013/06/01/ipython-notebook-javascript-python-communication/