HuggingFace Transformers For Text Generation with CTRL with Google Colab's free GPU

Question

我想使用 PyTorch-Transformers 使用 CTRL 测试 TextGeneration，然后再使用它进行微调。 但它不会像 GPT-2 和其他类似的语言生成模型那样提示任何东西。 我对此很陌生，被卡住了，无法弄清楚发生了什么。

这是我在 Colab 笔记本中遵循的程序，

!pip install transformers

!git clone https://github.com/huggingface/pytorch-transformers.git

!python pytorch-transformers/examples/run_generation.py \
    --model_type=ctrl \
    --length=100 \
    --model_name_or_path=ctrl \
    --temperature=0.2 \
    --repetition_penalty=1.2 \

这就是我运行脚本后得到的

02/10/2020 01:02:31 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-vocab.json from cache at /root/.cache/torch/transformers/a858ad854d3847b02da3aac63555142de6a05f2a26d928bb49e881970514e186.285c96a541cf6719677cfb634929022b56b76a0c9a540186ba3d8bbdf02bca42
02/10/2020 01:02:31 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-merges.txt from cache at /root/.cache/torch/transformers/aa2c569e6648690484ade28535a8157aa415f15202e84a62e82cc36ea0c20fa9.26153bf569b71aaf15ae54be4c1b9254dbeff58ca6fc3e29468c4eed078ac142
02/10/2020 01:02:31 - INFO - transformers.configuration_utils -   loading configuration file https://storage.googleapis.com/sf-ctrl/pytorch/ctrl-config.json from cache at /root/.cache/torch/transformers/d6492ca334c2a4e079f43df30956acf935134081b2b3844dc97457be69b623d0.1ebc47eb44e70492e0c20494a084f108332d20fea7fe5ad408ef5e7a8f2baef4
02/10/2020 01:02:31 - INFO - transformers.configuration_utils -   Model config CTRLConfig {
  "architectures": null,
  "attn_pdrop": 0.1,
  "bos_token_id": 0,
  "dff": 8192,
  "do_sample": false,
  "embd_pdrop": 0.1,
  "eos_token_ids": 0,
  "finetuning_task": null,
  "from_tf": false,
  "id2label": {
    "0": "LABEL_0"
  },
  "initializer_range": 0.02,
  "is_decoder": false,
  "label2id": {
    "LABEL_0": 0
  },
  "layer_norm_epsilon": 1e-06,
  "length_penalty": 1.0,
  "max_length": 20,
  "model_type": "ctrl",
  "n_ctx": 512,
  "n_embd": 1280,
  "n_head": 16,
  "n_layer": 48,
  "n_positions": 50000,
  "num_beams": 1,
  "num_labels": 1,
  "num_return_sequences": 1,
  "output_attentions": false,
  "output_hidden_states": false,
  "output_past": true,
  "pad_token_id": 0,
  "pruned_heads": {},
  "repetition_penalty": 1.0,
  "resid_pdrop": 0.1,
  "summary_activation": null,
  "summary_first_dropout": 0.1,
  "summary_proj_to_labels": true,
  "summary_type": "cls_index",
  "summary_use_proj": true,
  "temperature": 1.0,
  "top_k": 50,
  "top_p": 1.0,
  "torchscript": false,
  "use_bfloat16": false,
  "vocab_size": 246534
}

02/10/2020 01:02:31 - INFO - transformers.modeling_utils -   loading weights file https://storage.googleapis.com/sf-ctrl/pytorch/seqlen256_v1.bin from cache at /root/.cache/torch/transformers/c146cc96724f27295a0c3ada1fbb3632074adf87e9aef8269e44c9208787f8c8.b986347cbab65fa276683efbb9c2f7ee22552277bcf6e1f1166557ed0852fdf0
tcmalloc: large alloc 1262256128 bytes == 0x38b92000 @  0x7fe1900bdb6b 0x7fe1900dd379 0x7fe139843b4a 0x7fe1398455fa 0x7fe13bb7578a 0x7fe13bdbe30b 0x7fe13be05b37 0x7fe184c8cad5 0x7fe184c8d17b 0x7fe184c91160 0x7fe184ade496 0x551b15 0x5aa6ec 0x50abb3 0x50c5b9 0x508245 0x5096b7 0x595311 0x54a6ff 0x551b81 0x5aa6ec 0x50abb3 0x50c5b9 0x508245 0x509642 0x595311 0x54a6ff 0x551b81 0x5aa6ec 0x50abb3 0x50c5b9
tcmalloc: large alloc 1262256128 bytes == 0x19fdda000 @  0x7fe1900bdb6b 0x7fe1900dd379 0x7fe139843b4a 0x7fe1398455fa 0x7fe13bb7578a 0x7fe13bdbe30b 0x7fe13be05b37 0x7fe184c8cad5 0x7fe184c8d17b 0x7fe184c91160 0x7fe184ade496 0x551b15 0x5aa6ec 0x50abb3 0x50c5b9 0x508245 0x509642 0x595311 0x54a6ff 0x551b81 0x5aa6ec 0x50abb3 0x50d390 0x508245 0x509642 0x595311 0x54a6ff 0x551b81 0x5a067e 0x50d966 0x508245
^C

然后终止。 这可能是因为 GPU 问题吗？

任何形式的帮助表示赞赏。

Answer 1

解决方案是增加RAM。 因为我使用的是 Google Colab 的免费 G​​PU，所以我正在经历这个： GitHub 问题并发现这很有用： 解决方案

以下代码将使 Colab 中的会话崩溃并选择“获取更多 RAM”，这会将 RAM 增加到 25.51GB

d=[]
while(1):
  d.append('1')