为什么keras中的model.fit函数会显着增加RAM内存？

Question

I load my data with open_memmap function and it takes 5GB RAM memory. Then I compile the model which has params: 89,268,608 and it does not take any RAM memory. My batch size is 200 at the moment and the input image has shape (300,54,3).

My problem is when I call the model.fit function in keras my RAM memory increase from 5 GB to 24GB. My question is why?

When I try with different batch sizes nothing is changing and still 23 GB of RAM are occupied?

If somebody can explain me what is happening I would highly appreciate it,

Thanks!

Answer 1

Keras' fit method loads all the data into memory at once meaning changing your batch size will have no effect on the RAM it takes up. Have a look at using fit_generator which is designed for use with a large dataset.