[英]Slow training of BERT model Hugging face
I am training the binary classfier using BERT model implement in hugging face library我正在使用 BERT model implement in hugging face library 训练二进制分类器
training_args = TrainingArguments(
"deleted_tweets_trainer",
num_train_epochs = 1,
#logging_steps=100,
evaluation_strategy='steps',
remove_unused_columns = True
)
I am using Colab TPU still the training time is a lot, 38 hours for 60 hours cleaned tweets.我仍在使用 Colab TPU,训练时间仍然很多,38 小时清理推文 60 小时。
Is there any way to optimise the training?有什么办法可以优化训练?
You are currently evaluating every 500 steps and have a training and eval batch size of 8.您目前每 500 步评估一次,训练和评估批大小为 8。
Depending on your current memory consumption, you can increase the batch sizes (eval much more as training consumes more memory):根据您当前的 memory 消耗量,您可以增加批量大小(随着训练消耗更多内存,评估更多):
In case it matches your use case, you can also increase the steps after an evaluation is started;如果它符合您的用例,您还可以在评估开始后增加步骤;
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.