如何在 nltk POS 数据上应用 TFIDF ...？

Question

I applied nltk POS featurization to my reviews column in my panda's DataFrame. I got the new feature, but when I'm trying to apply TFIDF vectorization on that feature it shows error like

Error: While applying TFIDF Vectorizer

AttributeError: 'list' object has no attribute lower

I used below code for nltk POS

pure_df['pre_pro_plot_synopsis_POS'] = pos_tag_sents(pure_df['pre_pro_plot_synopsis'].apply(word_tokenize).tolist())

Can anyone help me to pass POS data to tfidf vectorizer?

Thank's in advance

Answer 1

Here it seems pos_tag_sents method created by you expects a string as paramenter, but here you are passing list as a parameter to it. So instead of this line: pos_tag_sents(pure_df['pre_pro_plot_synopsis'].apply(word_tokenize).tolist())

Try using apply method again and pass pos_tag_sents to it, like this:

pure_df['pre_pro_plot_synopsis_POS'] = pure_df['pre_pro_plot_synopsis'].apply(word_tokenize).apply(pos_tag_sents).tolist()

Hope it helps.