How to create and delete class instance in python for loop

Question

I am using a class to create a tensorflow model. Within a for loop, I am creating an instance which I must delete at the end of each iteration in order to free up memory. Deletion does not work and I am running out of memory. Here is a minimal example of what I tried:

import numpy as np

class tfModel(self, x):
   def __init__(self, x):
      ....

   def predict(self, x):
      ...
      return x_new


if __name__=="__main__":

   x = np.ones(100)
   for i in range(0, 3):
      model = tfModel(x)
      x = model.predict(x)
      del model

I've read in related questions that "del" only deletes a reference, not the class instance itself. But how can I ensure that all references are deleted and the instance can be garbage collected?

Answer 1

I think you are talking about two things:

1. the model itself. I assume your model can fit in your memory. Otherwise you could not run any prediction.
2. the data. If data is the problem, you should make a data generator with python so that not all the data exist in the memory at the same time. You should generate each example ( x ) or each batch of examples and feed them into the model to get prediction. The result could be serialized to disk when necessary if your memory cannot hold all results.

More concretely, something like this:

class tfModel(self):
   def __init__(self):
      ....

   def predict(self, x):
      ...
      return x_new

def my_x_generator():
  for x in range(100):
    yield x


THRESHOLD = 16

if __name__=="__main__":

   model = tfModel()
   my_result_buffer = []
   for x in my_x_generator():
      x_pred = model.predict(x)
      my_result_buffer.append(x_pred)
      if len(my_result_buffer) > THRESHOLD:
        ## serialize my_result_buffer to disk
        my_result_buffer = []

Also note that in my sample code above:

1. The constructor of tfModel should not depend on x . ( x is removed from __init__ ). Of course, you could use model parameters to initialize your model.
2. You should instantiate your model outside of the data loop. Eg, the model only needs to be instantiated once; the same model would be used to do predictions for all examples.

Answer 2

It seems to be a specific tensorflow problem. Using the module multiprocessing, one can generate processes within a for loop. The processes are closed when finished and the memory is freed.

I found this solution here: Clearing Tensorflow GPU memory after model execution