HuggingFace Transformers For Text Generation with CTRL with Google Colab's free GPU

Question

我想使用 PyTorch-Transformers 使用 CTRL 測試 TextGeneration，然后再使用它進行微調。 但它不會像 GPT-2 和其他類似的語言生成模型那樣提示任何東西。 我對此很陌生，被卡住了，無法弄清楚發生了什么。

這是我在 Colab 筆記本中遵循的程序，

!pip install transformers

!git clone https://github.com/huggingface/pytorch-transformers.git

!python pytorch-transformers/examples/run_generation.py \
    --model_type=ctrl \
    --length=100 \
    --model_name_or_path=ctrl \
    --temperature=0.2 \
    --repetition_penalty=1.2 \

這就是我運行腳本后得到的

02/10/2020 01:02:31 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-vocab.json from cache at /root/.cache/torch/transformers/a858ad854d3847b02da3aac63555142de6a05f2a26d928bb49e881970514e186.285c96a541cf6719677cfb634929022b56b76a0c9a540186ba3d8bbdf02bca42
02/10/2020 01:02:31 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-merges.txt from cache at /root/.cache/torch/transformers/aa2c569e6648690484ade28535a8157aa415f15202e84a62e82cc36ea0c20fa9.26153bf569b71aaf15ae54be4c1b9254dbeff58ca6fc3e29468c4eed078ac142
02/10/2020 01:02:31 - INFO - transformers.configuration_utils -   loading configuration file https://storage.googleapis.com/sf-ctrl/pytorch/ctrl-config.json from cache at /root/.cache/torch/transformers/d6492ca334c2a4e079f43df30956acf935134081b2b3844dc97457be69b623d0.1ebc47eb44e70492e0c20494a084f108332d20fea7fe5ad408ef5e7a8f2baef4
02/10/2020 01:02:31 - INFO - transformers.configuration_utils -   Model config CTRLConfig {
  "architectures": null,
  "attn_pdrop": 0.1,
  "bos_token_id": 0,
  "dff": 8192,
  "do_sample": false,
  "embd_pdrop": 0.1,
  "eos_token_ids": 0,
  "finetuning_task": null,
  "from_tf": false,
  "id2label": {
    "0": "LABEL_0"
  },
  "initializer_range": 0.02,
  "is_decoder": false,
  "label2id": {
    "LABEL_0": 0
  },
  "layer_norm_epsilon": 1e-06,
  "length_penalty": 1.0,
  "max_length": 20,
  "model_type": "ctrl",
  "n_ctx": 512,
  "n_embd": 1280,
  "n_head": 16,
  "n_layer": 48,
  "n_positions": 50000,
  "num_beams": 1,
  "num_labels": 1,
  "num_return_sequences": 1,
  "output_attentions": false,
  "output_hidden_states": false,
  "output_past": true,
  "pad_token_id": 0,
  "pruned_heads": {},
  "repetition_penalty": 1.0,
  "resid_pdrop": 0.1,
  "summary_activation": null,
  "summary_first_dropout": 0.1,
  "summary_proj_to_labels": true,
  "summary_type": "cls_index",
  "summary_use_proj": true,
  "temperature": 1.0,
  "top_k": 50,
  "top_p": 1.0,
  "torchscript": false,
  "use_bfloat16": false,
  "vocab_size": 246534
}

02/10/2020 01:02:31 - INFO - transformers.modeling_utils -   loading weights file https://storage.googleapis.com/sf-ctrl/pytorch/seqlen256_v1.bin from cache at /root/.cache/torch/transformers/c146cc96724f27295a0c3ada1fbb3632074adf87e9aef8269e44c9208787f8c8.b986347cbab65fa276683efbb9c2f7ee22552277bcf6e1f1166557ed0852fdf0
tcmalloc: large alloc 1262256128 bytes == 0x38b92000 @  0x7fe1900bdb6b 0x7fe1900dd379 0x7fe139843b4a 0x7fe1398455fa 0x7fe13bb7578a 0x7fe13bdbe30b 0x7fe13be05b37 0x7fe184c8cad5 0x7fe184c8d17b 0x7fe184c91160 0x7fe184ade496 0x551b15 0x5aa6ec 0x50abb3 0x50c5b9 0x508245 0x5096b7 0x595311 0x54a6ff 0x551b81 0x5aa6ec 0x50abb3 0x50c5b9 0x508245 0x509642 0x595311 0x54a6ff 0x551b81 0x5aa6ec 0x50abb3 0x50c5b9
tcmalloc: large alloc 1262256128 bytes == 0x19fdda000 @  0x7fe1900bdb6b 0x7fe1900dd379 0x7fe139843b4a 0x7fe1398455fa 0x7fe13bb7578a 0x7fe13bdbe30b 0x7fe13be05b37 0x7fe184c8cad5 0x7fe184c8d17b 0x7fe184c91160 0x7fe184ade496 0x551b15 0x5aa6ec 0x50abb3 0x50c5b9 0x508245 0x509642 0x595311 0x54a6ff 0x551b81 0x5aa6ec 0x50abb3 0x50d390 0x508245 0x509642 0x595311 0x54a6ff 0x551b81 0x5a067e 0x50d966 0x508245
^C

然后終止。 這可能是因為 GPU 問題嗎？

任何形式的幫助表示贊賞。

Answer 1

解決方案是增加RAM。 因為我使用的是 Google Colab 的免費 G​​PU，所以我正在經歷這個： GitHub 問題並發現這很有用： 解決方案

以下代碼將使 Colab 中的會話崩潰並選擇“獲取更多 RAM”，這會將 RAM 增加到 25.51GB

d=[]
while(1):
  d.append('1')