Extracting word features from BERT model

Question

So as you know, we can extract BERT features of word in a sentence. My question is, can we also extract word features that are not included in a sentence? For example, bert features of single words such as "dog", "human", etc.

Answer 1

The very first layer of BERT is a static embeddings table, so you can use it as any other embeddings table and embeddings for words (or more frequently subwords) that BERT uses input to the first self-attentive layer. The static embeddings are only comparable with each other, not with the standard contextual embeddings. If need them comparable embeddings, you can try passing single-word sentences to BERT, but note that this will be an embeddings of a single-word sentenece, not the word in general.

However, BERT is a sentence-level model that is supposed to get embeddings of words in context. It is not designed for static word embeddings, and methods specifically designed for static word embeddings (such as FastText) would certainly get better results.