使用带有本地模型的管道

Question

I am trying to use a simple pipeline offline. I am only allowed to download files directly from the web.

I went to https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english/tree/main and downloaded all the files in a local folder C:\\\\Users\\\\me\\\\mymodel

However, when I tried to load the model I get a strange error

from transformers import pipeline

classifier = pipeline(task= 'sentiment-analysis', 
                      model= "C:\\Users\\me\\mymodel",
                      tokenizer = "C:\\Users\\me\\mymodel")

ValueError: unable to parse C:\Users\me\mymodel\modelcard.json as a URL or as a local path

What is the issue here? Thanks!

Answer 1

Must be either of the two cases:

• You didn't download all the required files properly
• Folder path is wrong

FYI, I am listing out the required contents in the directory:

• config.json
• pytorch_model.bin/ tf_model.h5
• special_tokens_map.json
• tokenizer.json
• tokenizer_config.json
• vocab.txt

Answer 2

the solution was slightly indirect:

1. load the model on a computer with internet access
2. save the model with save_pretrained()
3. transfer the folder obtained above to the offline machine and point its path in the pipeline call

The folder will contain all the expected files.