我如何在 google colab 中解压缩预训练的 word2vec？

Question

I am trying to use pretrained word vectors of google, GoogleNews-vectors-negative300.bin.gz, in colab but i don't know how to unzip the file.

import gzip
f=gzip.open('gdrive/My Drive/Colab Notebooks/LAST/we/GoogleNews-vectors-negative300.bin.gz', 'rt')
file_content=f.read()

I tried to read the file directly using gzip but got the error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x94 in position 19: invalid start byte.

Answer 1

from gensim.models import KeyedVectors
word2vec = KeyedVectors.load_word2vec_format(EMBEDDING_FILE, binary=True)
x = word2vec.word_vec("test")

x would contain the vector for the word test

Snippet to download the word2Vec model:

EMBEDDING_FILE = '/root/input/GoogleNews-vectors-negative300.bin.gz'
!wget -P /root/input/ -c "https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz"

Reference: A publicly available Google Colab Notebook

Answer 2

There are two possible solutions (both of which I've tried. Even I'm working on the same problem):

1. Use encoding='iso8859'

2. Use KeyedVectors.load_word2vec_format(path of your file).

Answer 3

你可以使用这个：

!gunzip ./GoogleNews-vectors-negative300.bin.gz