Error in visualizing LDA Topic Model

Question

I want to interpret the topics in my lda topic model, so i am using pyldavis.. But somehow i can't get pyldavis to run. Here is the code:

import gensim
from gensim import corpora
from gensim.corpora import Dictionary


dictionary = corpora.Dictionary(lemmatized_list)
print(dictionary)
print(dictionary.token2id)
corpus = [dictionary.doc2bow(text) for text in lemmatized_list]
print(corpus)

ldamodel = gensim.models.ldamodel.LdaModel(corpus, num_topics=5, id2word = 
dictionary, passes=10)
print(ldamodel.print_topics(num_topics=5, num_words=3))

import pyLDAvis.gensim
pyLDAvis.enable_notebook()
pyLDAvis.gensim.prepare(ldamodel, corpus, dictionary)

then, after i reach to the last part of the code where i have to visualize using pyldavis, its showing the following error:

    ---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-16-a16bc334f38f> in <module>()
      1 import pyLDAvis.gensim
      2 pyLDAvis.enable_notebook()
----> 3 pyLDAvis.gensim.prepare(ldamodel, corpus, dictionary)
      4 term_ix = np.sort(topic_info.index.unique().values)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pyLDAvis\gensim.py in prepare(topic_model, corpus, dictionary, doc_topic_dist, **kwargs)
    108     """
    109     opts = fp.merge(_extract_data(topic_model, corpus, dictionary, doc_topic_dist), kwargs)
--> 110     return vis_prepare(**opts)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pyLDAvis\_prepare.py in prepare(topic_term_dists, doc_topic_dists, doc_lengths, vocab, term_frequency, R, lambda_step, mds, n_jobs, plot_opts, sort_topics)
    396 
    397    topic_info         = _topic_info(topic_term_dists, topic_proportion, term_frequency, term_topic_freq, vocab, lambda_step, R, n_jobs)
--> 398    token_table        = _token_table(topic_info, term_topic_freq, vocab, term_frequency)
    399    topic_coordinates = _topic_coordinates(mds, topic_term_dists, topic_proportion)
    400    client_topic_order = [x + 1 for x in topic_order]

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pyLDAvis\_prepare.py in _token_table(topic_info, term_topic_freq, vocab, term_frequency)
    265    # term-topic frequency table of unique terms across all topics and all values of lambda
    266    term_ix = topic_info.index.unique()
--> 267    term_ix.sort()
    268    top_topic_terms_freq = term_topic_freq[term_ix]
    269    # use the new ordering for the topics

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\indexes\base.py in sort(self, *args, **kwargs)
   2098 
   2099     def sort(self, *args, **kwargs):
-> 2100         raise TypeError("cannot sort an Index object in-place, use "
   2101                         "sort_values instead")
   2102 

TypeError: cannot sort an Index object in-place, use sort_values instead

Any suggestions on how to get this error sorted out...would be really helpful. Thanks!

Answer 1

This error has appeared before and has been identified as an incompatibility between Pandas and PyLDAvis in some versions.

Here they claim a specific version should fix it.

Duplicate of #76 its fixed in version 2.1.0. Install that specific version pip install pyldavis==2.1.0

I didn't check it though.