Python Regex句子过滤

Question

I'm trying to filter the following sentence

'I'm using C++ in high-tech applications!', said peter (in a confident way)

into its individual words to get

I'm using C++ in high-tech applications said peter in a confident way

what I have so far is

parsing=re.findall(r"\w+(?:[-']\w+)*|'|[-.(]+|\S\w*",text)
' '.join(w for w in parsing if w not in string.punctuation)

however this produces

I'm using C in high-tech applications said peter in a confident way

So 'C++' incorrectly turns into 'C' because '+' is in string.punctuation. Is there anyway I can modify the regex code to allow for '+''s not to be tokenized? Any alternative method to get the desired output would also be welcome, thanks!

Answer 1

Just use (\\w|\\+) instead of \\w . This will use both word characters and the plus sign.

Alternatively, you could use [a-zA-Z+] or ideally [\\w+] as suggested by Kyle Strand.

Answer 2

Similar to C0deH4cker's answer but slightly simpler, replace all instances of \\w with [\\w+] .

>>> parsing=re.findall(r"[\w+]+(?:[-'][\w+]+)*|'|[-.(]+|\S[\w+]*",text)
>>> parsing
["'", "I'm", 'using', 'C++', 'in', 'high-tech', 'applications', '!', "'", ',', 'said', 'peter', '(', 'in', 'a', 'confident', 'way', ')']
>>> ' '.join(w for w in parsing if w not in string.punctuation)
"I'm using C++ in high-tech applications said peter in a confident way"

Note that your original solution splits "C++" into three distinct tokens, so even excluding + from string.punctuation wouldn't have solved your problem:

>>> parsing=re.findall(r"\w+(?:[-']\w+)*|'|[-.(]+|\S\w*",text)
>>> parsing
["'", "I'm", 'using', 'C', '+', '+', 'in', 'high-tech', 'applications', '!', "'", ',', 'said', 'r', '(', 'in', 'a', 'confident', 'way', ')']