在 unicode 和文本上使用 re 拆分 python 字符串

Question

I'm trying to split a long string based on unicode and text (Chinese) punctuation. How do I do this?

def split1(s):
    temp1 = re.split(r"(;|:|•|。|；|：)", s)
    temp = re.split(u"([\u3002|\uFF01|\uFF1F])", temp1)
    i = iter(temp)

UPDATE: I'm hoping to split the string s based on regular text and unicode text.

Answer 1

You may use

def split1(s): 
    return re.split(ur"([\u3002\uFF01\uFF1F;:•。；：])", s)

It does not make sense to split the two patterns since the only purpose to use them is to tokenize a string into the chars that match the regex and those that do not.

The captured texts will also make part of the resulting list since the whole pattern is wrapped with a capturing group, see re.split docs :

If capturing parentheses are used in pattern, then the text of all groups in the pattern are also returned as part of the resulting list

Note the u prefix, too, it will tell Python 2.x to correctly handle Unicode code units in the string.