In TensorFlow, how to clear the GPU memory of an intermediate variable in a CNN model?

Question

I am just using TensorFlow to realise a CNN model. During the training process, there is an intermediate variable which occupies a large GPU memory and I want to clear the memory of this variable.

This variable is called 'rgb_concat', I just tried to use 'rgb_concat=[]' to clear its memory, not sure if it is useful in TensorFlow?

How could I achieve this in TensorFlow? Thanks in advance!

An intermediate variable called 'rgb_concat' which occupies a large GPU memory and I want to clear it and save GPU memory for other layers in a CNN model. How could I realise it in TensorFlow?

x = input_image
for j in range(n_sub_layers):
    nn = Conv2dLayer(x, j)     #
    rgb_concat.append(nn)
    x = nn
rgb_concat_sublayer = ConcatLayer([rgb_concat[0], rgb_concat[1]], concat_dim=3, name='rgb_concat_sublayer_{}_{}'.format(i,1))
for sub_layer in range(2, n_sub_layers): #Second 'for' loop!!!
        rgb_concat_sublayer = ConcatLayer([rgb_concat_sublayer, rgb_concat[sub_layer]], concat_dim=3, name='rgb_concat_sublayer_{}_{}'.format(i,sub_layer))

Since I do not need 'rgb_concat' after the second 'for' loop any more, it should be cleared after 'for' loop.

Answer 1

Have you tried the del keyword?

del rgb_concat

You could also just set the variable to None.

rgb_concat = None