Python 正则表达式：查找所有正则表达式以将文本字符串匹配到棘手的规范并将最终结果放在单词列表中

Question

I have a string:

sample_input = """
This film is based on Isabel Allende's not-so-much-better novel. I hate Meryl
Streep and Antonio Banderas (in non-Spanish films), and the other actors,
including Winona, my favourite actress and Jeremy Irons try hard to get over
such a terrible script.

I want to apply regex to it so that it can produce desired output:

['this', 'film', 'is', 'based', 'on', 'isabel', "allende's", 'not-so', 'much-better', 'novel', 'i', 'hate', 'meryl', 'streep', 'and', 'antonio', 'banderas', 'in', 'non-spanish', 'films', 'and', 'the', 'other', 'actors', 'including', 'winona', 'my', 'favourite', 'actress', 'and', 'jeremy', 'irons', 'try', 'hard', 'to', 'get', 'over', 'such', 'a', 'terrible', 'script']

I want to create a list of words (all lowercase) with the following rules:

1. a word has to begin and end with single letter or number.
2. can only have one hyphen (-) or one apostraphe (') in a word
3. if violate 1 or 2 then it's a new word

** Please see desired output for details.

Note that the regex can only allow one hyphen or one apostrophe in a word, but no more than one of these per word.

I tried the following code:

sample_output_regex = re.findall(r'[a-zA-Z0-9]*[-]?|[\']?[a-zA-Z0-9]*', sample_input.lower())

But the output is pretty off:

['', 'this', '', 'film', '', 'is', '', 'based', '', 'on', '', 'isabel', '', 'allende', '', "'s", '', 'not-', 'so-', 'much-', 'better', '', 'novel', '', '', 'i', '', 'hate', '', 'meryl', '', 'streep', '', 'and', '', 'antonio', '', 'banderas', '', '', 'in', '', 'non-', 'spanish', '', 'films', '', '', '', 'and', '', 'the', '', 'other', '', 'actors', '', '', 'including', '', 'winona', '', '', 'my', '', 'favourite', '', 'actress', '', 'and', '', 'jeremy', '', 'irons', '', 'try', '', 'hard', '', 'to', '', 'get', '', 'over', '', 'such', '', 'a', '', 'terrible', '', 'script', '', '', '']

In an effort to get better at regex, I would like to know where my regex code is off. How do I change it to get my desired output. Details would be appreciated. For instance, why are the spaces getting pulled through as '' when my regex doesn't ask to match spaces?

Answer 1

About the pattern:

You get the empty entries as all the parts in your pattern [a-zA-Z0-9]*[-]?|[\']?[a-zA-Z0-9]* are optional.

Due to the alternation | this for example not-so will not be a single match, as the part after the - will not be matched.

---

You might use an approach like:

\b[a-zA-Z0-9]+(?:[-'][a-zA-Z0-9]+)?\b

The pattern matches

• \b A word boundary
• [a-zA-Z0-9]+ Match 1+ times any of the listed ranges
• (?: Non capture group
  • [-'][a-zA-Z0-9]+ Match a single - or ' and 1+ of the listed ranges
• )? Close the group and make it optional
• \b A word boundary

regex demo

Then you can turn all the matches into lower cases ones.

import re

sample_input = """
This film is based on Isabel Allende's not-so-much-better novel. I hate Meryl
Streep and Antonio Banderas (in non-Spanish films), and the other actors,
including Winona, my favourite actress and Jeremy Irons try hard to get over
such a terrible script."""

res = [x.lower() for x in re.findall(r"\b[a-zA-Z0-9]+(?:[-'][a-zA-Z0-9]+)?\b", sample_input)]
print(res)

Output

['this', 'film', 'is', 'based', 'on', 'isabel', "allende's", 'not-so', 'much-better', 'novel', 'i', 'hate', 'meryl', 'streep', 'and', 'antonio', 'banderas', 'in', 'non-spanish', 'films', 'and', 'the', 'other', 'actors', 'including', 'winona', 'my', 'favourite', 'actress', 'and', 'jeremy', 'irons', 'try', 'hard', 'to', 'get', 'over', 'such', 'a', 'terrible', 'script']