How to resolve ResourceExhaustedError in google colab

Question

I am using a roberta for question answering model for tweet sentiment extraction problem on google colab.

but the model is unable to train as I get a Resourceexhaustederror;

see full error:

ResourceExhaustedError:  OOM when allocating tensor with shape[32,16,128,64] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
     [[node model/tf_roberta_model/roberta/encoder/layer_._17/attention/self/transpose (defined at /usr/local/lib/python3.7/dist-packages/transformers/models/roberta/modeling_tf_roberta.py:218) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
 [Op:__inference_train_function_112984]...

see model here:

ids = Input((MAX_LEN,), dtype=tf.int32)
att = Input((MAX_LEN,), dtype=tf.int32)

bert_model = TFRobertaModel.from_pretrained('roberta-large')

x = bert_model(ids, attention_mask= att)

x1 = Dropout(0.1)(x[0])
x1 = Conv1D(1,1)(x1)
x1 = Flatten()(x1)
x1 = Activation('softmax')(x1)


x2 = Dropout(0.1)(x[0])
x2 = Conv1D(1,1)(x2)
x2 = Flatten()(x2)
x2 = Activation('softmax')(x2)


model = Model(inputs = [ids, att], outputs = [x1, x2])

any help in resolving this error would be appreciated.

Answer 1

In my experience, you can either use the Gradient Accumulation technique. Or, a much better option if you can manage to use Google Colab Pro . According to the doc

With Colab Pro you get priority access to high-memory VMs. These VMs generally have double the memory of standard Colab VMs, and twice as many CPUs. You will be able to access a notebook setting to enable high-memory VMs once you are subscribed. Additionally, you may sometimes be automatically assigned a high-memory VM when Colab detects that you are likely to need it. Colab Pro VMs also generally come with double the disk of standard Colab VMs. Resources are not guaranteed, though, and there are usage limits for high memory VMs.

In the free version of Colab, the high-memory preference is not available, and users are rarely automatically assigned high memory VMs.

These transformer models are memory hunger, so it's very convenient to use Colab Pro. However, you can also use the TPU accelerator that also available in Colab but note that it's much slower than the Kaggle TPU.