检查字符串（或拆分字符串）是否包含列表中的任何单词

Question

I have a Twitter bot that needs to ignore tweets that contain certain blacklisted words.

This works, but only if the words in the tweet are exactly as they're seen in the list of blacklisted words.

timeline = filter(lambda status: not any(word in status.text.split() for word in wordBlacklist), timeline)

I want to make sure that tweets can't bypass this by putting symbols or adding additional characters around a word, such as bypassing blacklisted word "face" by appending "book" to the end of it, like so "facebook".

How do I do this in a way that fits within my filter's lambda?

Answer 1

You can make use of re here.

import re
timeline = filter(lambda status: not any(re.findall(r"[a-zA-Z0-9]*"+word+r"[a-zA-Z0-9]*",status.text) for word in wordBlacklist), timeline)

You can also use re.escape() over word if word can contain some escape characters

If you expect symbols as well ,try

timeline = filter(lambda status: not any(re.findall(r"\S*"+word+r"\S*",status.text) for word in wordBlacklist), timeline)

Answer 2

You can construct a regular expression based on the blacklist:

from itertools import ifilterfalse
import re

wordBlacklist = ['face', 'hello']

r = re.compile('|'.join(map(re.escape, wordBlacklist)))

...
timeline = list(ifilterfalse(lambda status: r.search(status.text), timeline))

Answer 3

Instead of filter, you can use a list comprehension, which is the same idea with a slightly different syntax, and then use regular expressions for the filtering, as your example is beyond the capabilities of string operations:

import re
blacklist = re.compile('face|friend|advertisement')
timeline = [word for word in status.split() if not blacklist.search(word)]
# filter version of this command:
timeline = filter(lambda word: not blacklist.search(word), status.split())

Now timeline will return a list of words that don't have any match to your blacklist within them, so "facebook" would be blocked because it matches "face", "friendly" would be blocked because it contains "friend", etc. However, you are going to need to get fancier for things like "f*acebook" or other tricks-- these would bypass the filter currently. Try out regex and get comfortable with them, and you can really make pretty fancy filters. Here is a good practice site for regex.