在计算单词频率时是否可以将两个单词加在一起？ 蟒蛇

Question

top_N = 100

words = review_tip['user_tip'].dropna()
words = words.astype(str)
words = words.str.replace('[{}]'.format(string.punctuation), '')
words = words.str.lower().apply(lambda x: ' '.join([word for word in x.split() if word not in (stopwords)]))

# replace '|'-->' ' and drop all stopwords
words = words.str.lower().replace([r'\|', RE_stopwords], [' ', ''], regex=True).str.cat(sep=' ').split()

# generate DF out of Counter
rslt = pd.DataFrame(Counter(words).most_common(top_N),
                    columns=['Word', 'Frequency']).set_index('Word')
print(rslt)
plt.clf()
# plot
rslt.plot.bar(rot=90, figsize=(16,10), width=0.8)
plt.show()
            Frequency
Word                 
great           17069

food            16381

good            12502

service         11342

place           10841

best             9280

get              7483

love             7042

amazing          5043

try              4945

time             4810

go               4594

dont             4377

As you can see the words are singular which is something I can use, but is it possible to take like two words that couldve been used together a lot?

For example getting

dont go (this could be for 100 times)

instead of getting it separate

dont 100

go 100

Answer 1

This will generate bi-grams, is this what you are looking for:

bi_grams = zip(words, words[1:])

I generates tuples which is fine to use in the counter but you could also easily tweak the code to use ' '.join((a, b)) .