在 Colab 中运行我的代码时，为什么我的运行时内存不足？

Question

IMAGE_SIZE = 128
IMAGE_CHANNELS = 3

ImageFile.LOAD_TRUNCATED_IMAGES = True

for filename in tdqm(os.listdir(images_path)):
  path = os.path.join(images_path, filename)
  image = Image.open(path).resize((IMAGE_SIZE, IMAGE_SIZE), Image.NEAREST)
  
  if (np.asarray(image).size != 49152):
    bad = bad + 1
    print(bad)
    print(path)
  
  picarray = np.asarray(image)
  #picarray = (picarray>>16).astype(np.int16)
  
  training_data.append(picarray)

training_data = np.reshape(training_data, (-1, IMAGE_SIZE, IMAGE_SIZE, IMAGE_CHANNELS))
#print(training_data.)

training_data = training_data / 127.5 - 1

print('saving file...')
np.save('/content/drive/MyDrive/Art/cubism_data.npy', training)

When I run the code above my session uses up all its ram and I am unsure why. Any help would be appreciated.

Answer 1

You are uploading all your images to the RAM. This could be the reason for the error.

I don't really understand what is the end goal of this script, I am assuming that you want to upload a training set for a learning process. If that is true you can use different techniques to only uploading a batch of images at a time. You can use the tensorflow Dataset object that will take care of the work for you.

However if you must open all the images Then you should consider compressing them or resizing them.