How to use example scripts from git repo when installing with pip

Question

This might be a very strange question for most of you, but in this case I am grateful for an easy explanation. What confuses me is the following. Let's say I have a git repository such as the following:

https://github.com/cedrickchee/pytorch-pretrained-BERT

In the README they say, I can either install the repository with pip or from source. Within the repo, there are certain .py scripts that I want to use. What creates confusion to me is: How can I access those scripts when installing the repository with pip? I am talking about scripts like these:

python3 simple_lm_finetuning.py 
--train_corpus my_corpus.txt 
--bert_model bert-base-uncased 
--do_lower_case 
--output_dir finetuned_lm/
--do_train

Is it necessary to install the repository from source in this case? Thanks for your help!

Answer 1

Installing from remote package repository ( pip install pytorch-pretrained-bert ) and installing from source (in source dir, pip install. ), has almost the same effect. Library is installed locally so it can be imported with import pytorch_pretrained_bert . The difference is that some changes can be already available in source and not released in package. In this case current package repository version can be checked here As said there were few changes in code sice last release.

What you also need to know is that installed package includes only files from pytorch_pretrained_bert module so users only download sufficient files. In result documentation, tests and docker files are excluded.

After installing package, you can access:

from pytorch_pretrained_bert import optimization

But you can't access:

from pytorch_pretrained_bert import examples

Full example is often included in one file, so you can just paste/copy it . You can also git clone whole code repository and run it directly from local directory.

Answer 2

Those scripts are examples. They are not installed even if you install from the git repository. To access them you need to clone the repository and copy the scripts out of examples directory.