如何使用Github和CheckPoint文件中预先训练的Tensorflow模型进行推理

Question

I would like to collect text embeddings from the model in this github

https://github.com/dmis-lab/biobert

Under installation, all it says is

To use BioBERT, we need pre-trained weights of BioBERT, which you can download from Naver GitHub repository for BioBERT pre-trained weights. Make sure to specify the versions of pre-trained weights used in your works. Also, note that this repository is based on the BERT repository by Google.

All the fine-tuning experiments were conducted on a single TITAN Xp GPU machine which has 12GB of RAM. The code was tested with Python2 and Python3 (We used Python2 for experiments). You might want to install java to use official evaluation script of BioASQ. See requirements.txt for other details.

I can download their checkpoint file and load it using something like

with tf.Session(graph=graph) as session:

   saver.restore(session, 'BioBert.ckpt' )

And install their github using something like

!test -d bioBert_repo|| git clone https://github.com/dmis-lab/biobert bioBert_repo

But how do I get the embedding from a text input. The instructions say it's based on BERT, but for BERT all we have to do is import the tf.hub model

bert_module = hub.Module(
 "https://tfhub.dev/google/bert_uncased_L-12_H-768_A-12/1",
 trainable=False)

then put the tokenized sentence into that

bert_embedding= bert_module(inputs=tokenized_sentence, signature="tokens", as_dict=True)[
       "pooled_output"
   ]

I'm guessing there's a similar method where I can install the github and load the weights but I can't seem to find it.

Answer 1

You should see example from extract_features.py. I suppose that BIOBert don't use tf.hub.