为什么 Biterm 主题 Model (BTM) 返回一致性分数 -100？

Question

I am using biterm.cbtm library to train a topic model of about 2500 short posts. When BTM finishes, I get the following 10 topics, along with the topic coherence value as shown in this picture: https://ibb.co/Kqy992H

I am trying to understand what those negative coherence values mean and why they are so low. I read a lot of related research and I couldn't find one paper that explains the range of the coherence value. Also, most of the papers where about LDA coherence value, as BTM is not well documented.

Does anyone know the range/meaning of the coherence value I am getting? Why is coherence between -76 and -111?

You can see my code below:

 from sklearn.feature_extraction.text import CountVectorizer from biterm.utility import vec_to_biterms import numpy as np import pyLDAvis from biterm.cbtm import oBTM from sklearn.feature_extraction.text import CountVectorizer from biterm.utility import vec_to_biterms, topic_summuary # helper functions import pickle import pandas as pd from numpy import array import numpy as np import logging import pyLDAvis.gensim import json import warnings import pickle import pandas as pd import re warnings.filterwarnings('ignore') # To ignore all warnings that arise here to enhance clarity from gensim.models.coherencemodel import CoherenceModel from gensim.models.ldamodel import LdaModel from gensim.corpora.dictionary import Dictionary from nltk.stem.wordnet import WordNetLemmatizer from nltk.tokenize import RegexpTokenizer import matplotlib.pyplot as plt from gensim import corpora, models from gensim.models import Phrases import time def docs_preprocessor(docs): tokenizer = RegexpTokenizer(r'\w+') for idx in range(len(docs)): docs[idx] = re.sub(r'(https?:\/\/)(\s)*(www\.)?(\s)*((\w|\s)+\.)*([\w\-\s]+\/)*([\w\-]+)((\?)?[\w\s]*=\s*[\w\%&]*)*', '', docs[idx]) docs[idx] = docs[idx].lower() # Convert to lowercase. if len(docs[idx]) < 50: continue docs[idx] = tokenizer.tokenize(docs[idx]) # Split into words. # Remove numbers, but not words that contain numbers. docs = [[token for token in doc if not token.isdigit()] for doc in docs] # Remove words that are only one character. docs = [[token for token in doc if len(token) > 3] for doc in docs] # Lemmatize all words in documents. lemmatizer = WordNetLemmatizer() docs = [[lemmatizer.lemmatize(token) for token in doc] for doc in docs] return docs colnames = ['post'] with open('cleantext.p', 'rb') as handle: dict = pickle.load(handle) dict['text'] = list(filter(None.__ne__, dict['text'])) print("Total posts: " + str(len(dict['text']))) p_df = pd.DataFrame.from_dict(dict)#, skiprows = lambda x: logic(x)) docs = array(p_df['text']) print("ALL DOCUMENTS: " + str(len(docs))) docs = docs_preprocessor(docs) outfile = open("posts.txt", "w+") total_docs = 0 for sentence in docs: if len(sentence) < 3: continue else: total_docs += 1 for word in sentence: result = ''.join([i for i in word if not i.isdigit()]) outfile.write(result + " ") outfile.write("\n") outfile.close() print("Total docs: " + str(total_docs)) print("Reading sentences. . .") texts = open('posts.txt', 'r').read().splitlines() clear_text = "" for item in texts: clear_text = clear_text + " " + item vec = CountVectorizer(stop_words='english') print("Building Vectors. . .") X = vec.fit_transform(texts).toarray() print("Building Vocabulary. . .") vocab = np.array(vec.get_feature_names()) biterms = vec_to_biterms(X) print("BTM modelling. . .") btm = oBTM(num_topics=10, V=vocab) print("\n\n Train Online BTM..") btm.fit(biterms, iterations=100) topics = btm.transform(biterms) print("\n\n Topic coherence..") topic_summuary(btm.phi_wz.T, X, vocab, 10) #I am getting a weird error about pyLDAvis here. Why? print("\n\n Visualize Topics..") vis = pyLDAvis.prepare(btm.phi_wz.T, topics, np.count_nonzero(X, axis=1), vocab, np.sum(X, axis=0)) pyLDAvis.save_html(vis, 'btm.html')

Answer 1

I guess we can refer to the idea in coherence understanding of LDA since the formula should be the same.:)

You may take a look at the interpretation here: :) Negative Values: Evaluate Gensim LDA with Topic Coherence