使用 split() 在各种标点符号处分割字符串

Question

I'm trying to divide a string into words, removing spaces and punctuation marks.

I tried using the split() method, passing all the punctuation at once, but my results were incorrect:

>>> test='hello,how are you?I am fine,thank you. And you?'
>>> test.split(' ,.?')
['hello,how are you?I am fine,thank you. And you?']

I actually know how to do this with regexes already, but I'd like to figure out how to do it using split() . Please don't give me a regex solution.

Answer 1

If you want to split a string based on multiple delimiters, as in your example, you're going to need to use the re module despite your bizarre objections, like this:

>>> re.split('[?.,]', test)
['hello', 'how are you', 'I am fine', 'thank you', ' And you', '']

It's possible to get a similar result using split , but you need to call split once for every character, and you need to iterate over the results of the previous split. This works but it's ugly:

>>> sum([z.split() 
... for z in sum([y.split('?') 
... for y in sum([x.split('.') 
... for x in test.split(',')],[])], [])], [])
['hello', 'how', 'are', 'you', 'I', 'am', 'fine', 'thank', 'you', 'And', 'you']

This uses sum() to flatten the list returned by the previous iteration.

Answer 2

这是我能想到的不使用 re 模块的最佳方法：

"".join((char if char.isalpha() else " ") for char in test).split()

Answer 3

由于你不想使用 re 模块，你可以使用这个：

 test.replace(',',' ').replace('.',' ').replace('?',' ').split()

Answer 4

A modified version of larsks' answer, where you don't need to type all punctuation characters yourself:

import re, string

re.split("[" + string.punctuation + "]+", test)
['hello', 'how are you', 'I am fine', 'thank you', ' And you', '']

Answer 5

You can write a function to extend usage of .split() :

def multi_split(s, seprators):
    buf = [s]
    for sep in seprators:
        for loop, text in enumerate(buf):
            buf[loop:loop+1] = [i for i in text.split(sep) if i]
    return buf

And try it:

>>> multi_split('hello,how are you?I am fine,thank you. And you?', ' ,.?') ['hello', 'how', 'are', 'you', 'I', 'am', 'fine', 'thank', 'you', 'And', 'you']

This will be clearer and can be used in other situations.

Answer 6

Apologies for necroing - this thread comes up as the first result for non-regex splitting of a sentence. Seeing as I had to come up with a non Python-specific method for my students, and that this thread didn't answer my question, I thought I would share just in case.

The point of the code is to use no libraries (and it's quick on large files):

sentence = "George Bernard-Shaw was a fine chap, I'm sure - who can really say?"
alpha = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
words = []
word = ""
mode = 0
for ch in sentence:
    if mode == 1:
        words.append(word)
        word = ""
        mode = 0
    if ch in alpha or ch == "'" or ch == "-":
        word += ch
    else:
        mode = 1
words.append(word)
print(words)

Output:

['George', 'Bernard-Shaw', 'was', 'a', 'fine', 'chap', "I'm", 'sure', '-', 'who', 'can', 'really', 'say']

I have literally just written this in about half an hour so I'm sure the logic could be cleaned up. I have also acknowledged that it may require additional logic to deal with caveats such as hyphens correctly, as their use is inconsistent compared to something like an inverted comma. Is there any module, indeed, that can do this correctly anyway?

Answer 7

A simple way to keep punctuation or other delimiters is:

import re

test='hello,how are you?I am fine,thank you. And you?'

re.findall('[^.?,]+.?', test)

Result:

['hello,', 'how are you?', 'I am fine,', 'thank you.', ' And you?']

maybe this can help someone.