Loading image dataset into memory with PIL (python) consumes too much memory

Question

I am using Google Colab (13 Gb RAM) and I am trying to load into memory 8000 JPG images (512x512), that on average are 150Kb each one.

I would expect that with all this would consume no more than 1.5GB in RAM, but actually consumes it all and Google Colab crashes.

What am I missing?

images = []
files = os.listdir(IMAGES_PATH)
for f in files:
  temp_image = Image.open(IMAGES_PATH + f)                
  temp = np.array(temp_image.convert('RGB'), dtype='float32') / 255
  images.append(temp)

Answer 1

You are using:

512*512*8000 this is the matrix you get after your calculation, and a float is using 8kbytes, which means your memory use is: 512*512*8000*8 = 11,11GB

The original size of your image does not matter for this calculation, python is storing his on memory.