Print GPU and CPU usage in TensorFlow

Question

I'm running some TensorFlow examples on Google Colab (so I can have a GPU), like this one.

Is there a way to print the CPU and GPU usage , in the code, for every training step, in order to see how the GPU is used and the performance difference between CPU-only and GPU?

In a standard environment maybe I could use nvidia-smi to track the GPU usage, but with the notebook, I can only run a cell at a time.

Thanks

Answer 1

There is a snippet code that I scraped from Internet. You can run the printm function whenever you want.

# memory footprint support libraries/code
!ln -sf /opt/bin/nvidia-smi /usr/bin/nvidia-smi
!pip install gputil
!pip install psutil
!pip install humanize
import psutil
import humanize
import os
import GPUtil as GPU
GPUs = GPU.getGPUs()
# XXX: only one GPU on Colab and isn’t guaranteed
gpu = GPUs[0]
def printm():
 process = psutil.Process(os.getpid())
 print("Gen RAM Free: " + humanize.naturalsize( psutil.virtual_memory().available ), " | Proc size: " + humanize.naturalsize( process.memory_info().rss))
 print("GPU RAM Free: {0:.0f}MB | Used: {1:.0f}MB | Util {2:3.0f}% | Total {3:.0f}MB".format(gpu.memoryFree, gpu.memoryUsed, gpu.memoryUtil*100, gpu.memoryTotal))
printm()

Here is the output from my Google Colab:

Gen RAM Free: 12.8 GB  | Proc size: 155.7 MB
GPU RAM Free: 11441MB | Used: 0MB | Util   0% | Total 11441MB

Answer 2

You need to start a thread that print this for you. While this thread is running you will see the output only when other cell is running.

The code:

!ln -sf /opt/bin/nvidia-smi /usr/bin/nvidia-smi
!pip install gputil
!pip install psutil
!pip install humanize
import psutil
import humanize
import os, time
import GPUtil as GPU

GPUs = GPU.getGPUs()
# XXX: only one GPU on Colab and isn’t guaranteed
gpu = GPUs[0]
def worker():
  while True:
    process = psutil.Process(os.getpid())
    print("Gen RAM Free: " + humanize.naturalsize( psutil.virtual_memory().available `enter code here`), " I Proc size: " + humanize.naturalsize( process.memory_info().rss))
    print("GPU RAM Free: {0:.0f}MB | Used: {1:.0f}MB | Util {2:3.0f}% | Total {3:.0f}MB".format(gpu.memoryFree, gpu.memoryUsed, gpu.memoryUtil*100, gpu.memoryTotal))
    time.sleep(6)

import threading
t = threading.Thread(target=worker, name='Monitor')
t.start()

The test: