如何在我的 python 代码中启用 Intel Extension for Pytorch(IPEX)？

Question

I would like to use Intel Extension for Pytorch in my code to increase overall performance. Referred this GitHub( https://github.com/intel/intel-extension-for-pytorch ) for installation.

Currently, I am trying out a hugging face summarization PyTorch sample( https://github.com/huggingface/transformers/blob/master/examples/pytorch/summarization/run_summarization.py ). Below is the trainer API used for training.

 # Initialize our Trainer trainer = Seq2SeqTrainer( model=model, args=training_args, train_dataset=train_dataset if training_args.do_train else None, eval_dataset=eval_dataset if training_args.do_eval else None, tokenizer=tokenizer, data_collator=data_collator, compute_metrics=compute_metrics if training_args.predict_with_generate else None, )

I am not aware of enabling Ipex in this code. Can anyone help me with this?

Thanks in Advance!

Answer 1

For enabling Intel Extension for Pytorch you just have to give add this to your code,

import intel_extension_for_pytorch as ipex

Importing above extends PyTorch with optimizations for extra performance boost on Intel hardware

After that you have to add this in your code

model = model.to(ipex.DEVICE)

Answer 2

First, you will need to subclass the Trainer object and create an custom optimizer as described in the Hugging Face docs

The APIs for using intel_extension_for_pytorch has changed a bit, to use the library, you just have to do:

import intel_extension_for_prytorch as ipex

model, optimizer = ipex.optimize(model, optimizer=optimizer)

Answer 3

Currently, Transformers 4.21 has support IPEX. IPEX Graph Optimization with JIT-mode

python run_qa.py
    --model_name_or_path csarron/bert-base-uncased-squad-v1 \
    --dataset_name squad \
    --do_eval \
    --max_seq_length 384 \
    --doc_stride 128 \
    --output_dir /tmp/ \
    --no_cuda \
    --use_ipex \
    --jit_mode_eval