正则表达式连接由空格和连字符分隔的单词

Question

My string is quite messy and looks something like this:

s="I'm hope-less and can -not solve this pro- blem on my own. Wo - uld you help me?"

I'd like to have the hyphen (& sometimes whitespace) stripped words together in one list.. Desired output:

list = ['I'm','hopeless','and','cannot','solve','this','problem','on','my','own','.','Would','you','help','me','?']

I tried a lot of different variations, but nothing worked..

rgx = re.compile("([\w][\w'][\w\-]*\w)") s = "My string'" rgx.findall(s)

Answer 1

Here's one way:

[re.sub(r'\s*-\s*', '', i) for i in re.split(r'(?<!-)\s(?!-)', s)]

# ["I'm", 'hopeless', 'and', 'cannot', 'solve', 'this', 'problem', 'on', 'my', 'own.', 'Would', 'you', 'help', 'me?']

Two operations here:

1. Split the text based on whitespaces without hyphens using both negative lookahead and negative lookbehind.

2. In each of the split word, replace the hyphens with possible whitespaces in front or behind to empty string.

You can see the first operation's demo here: https://regex101.com/r/ayHPvY/2

And the second: https://regex101.com/r/ayHPvY/1

Edit: To get the . and ? to be separated as well, use this instead:

[re.sub(r'\s*-\s*','', i) for i in re.split(r"(?<!-)\s(?!-)|([^\w\s'-]+)", s) if i]

# ["I'm", 'hopeless', 'and', 'cannot', 'solve', 'this', 'problem', 'on', 'my', 'own', '.', 'Would', 'you', 'help', 'me', '?']

The catch was also splitting the non-alphabets, non-whitespace and not hyphens/apostrophe. The if i is necessary as the split might return some None items.

Answer 2

Quick, non-regex way to do it would be

''.join(map(lambda s: s.strip(), s.split('-'))).split()

that is split on hyphens, strip of additional whitespaces, join back into string and split on space, this however doesn't separate dot or question marks.

Answer 3

How about this:

>>> s
"I'm hope-less and can -not solve this pro- blem on my own. Wo - uld you help me
?"
>>> list(map(lambda x:re.sub(' *- *','',x), filter(lambda x:x, re.split(r'(?<!-) +(?!-)|([.?])',s))))
["I'm", 'hopeless', 'and', 'cannot', 'solve', 'this', 'problem', 'on', 'my', 'own', '.', 'Would', 'you', 'help', 'me', '?']

Above used a simple space ' ' , but use \s is better:

list(map(lambda x:re.sub('\s*-\s*','',x), filter(lambda x:x, re.split(r'(?<!-)\s+(?!-)|([.?])',s))))

(?<!-)\s+(?!-) means spaces that don't have - before or after.
[.?] means single . or ? .

re.split(r'(?<!-)\s+(?!-)|([.?])',s) will split the string accordingly, but will have some None and empty string '' inside:

["I'm", None, 'hope-less', None, 'and', None, 'can -not', None, 'solve', None, 'this', None, 'pro- blem', None, 'on', None, 'my', None, 'own', '.', '', None, 'Wo - uld', None, 'you', None, 'help', None, 'me', '?', '']

This result was directly feed to filter to remove None and '' , and then feed to map to remove space and - inside each word.