[英]How to extract specific words from pieces of text, using a dictionary of words in categories?
我想从数据框中的文本中提取特定的单词。 这些词我已经输入到字典的列表中,它们属于某些类别(键)。 从这里我想创建与存储单词的类别相对应的列。 与往常一样,最好通过示例来说明:
我有一个数据框:
df = pd.DataFrame({'Text': ["This car is fast, agile and large and wide", "This wagon is slow, sluggish, small and compact with alloy wheels"]} )
创建表:
Text
0 This car is fast, agile and large and wide
1 This wagon is slow, sluggish, small and compact with alloy wheels
以及我想从中提取的类别中的单词词典。 这些单词都是没有符号的自然语言单词,并且可以包含短语,例如本例中的“合金车轮””(这不一定是字典,我只是觉得这是最好的方法):
myDict = {
"vehicle": ["car", "wagon"],
"speed": ["fast", "agile", "slow", "sluggish"],
"size": ["large", "small", "wide", "compact"]
"feature": ["alloy wheels"]
}
从这里我想创建一个看起来像这样的表:
| Text | vehicle | speed | size | feature |
| ----------------------------------------------------------------- | ------- | -------------- | -------------- | ------------ |
| This car is fast, agile and large and wide | car | fast, agile | large, wide | NaN |
| This wagon is slow, sluggish, small and compact with allow wheels | wagon | slow, sluggish | small, compact | alloy wheels |
提前为帮助干杯! 很想使用正则表达式,但欢迎任何解决方案!
有很多方法可以解决这个问题。 我可能开始的一种方法是:定义一个 function 如果它们与您的句子匹配,则返回一个单词列表。
def get_matching_words(sentence, category_dict, category):
matching_words = list()
for word in category_dict[category]:
if word in sentence.split(" "):
matching_words.append(word)
return matching_words
然后,您想将此 function 应用于您的 pandas dataframe。
df["vehicle"] = df["Text"].apply(lambda x: get_matching_words(x, "vehicle", my_dict))
df["speed"] = df["Text"].apply(lambda x: get_matching_words(x, "speed", my_dict))
这里唯一要添加的是将列表连接成一个字符串,而不是返回一个列表。
def get_matching_words(sentence, category_dict, category):
matching_words = list()
for word in category_dict[category]:
if word in sentence:
matching_words.append(word)
return ",".join(matching_words)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.