使用带有特殊符号的re解析语法Python

Question

I am trying to parse a file like this:

while (true){
    print("hello world")
}

while this is not Python syntax, i am using python for the parsing. My code is:

        words = []
        for line in lines: #line holds array of the above lines
            words += re.sub("[\s]", " ", line).split()

my result is:

['while', '(true){', 'print("hello', 'world")', '}']

which is cool since I only used re with a [\\s] regex, but how would I get a result like this:

['while', '(', 'true', ')', '{'....]

Where I get all symbols (lets assume I have a string that contains them one after the other, for example symbols = '(){}:,=+-') ?

Answer 1

You can use re.split with a group to get the split text and the split characters.

For instance, a symbol can be matched with the r'\\W+' RegEx.

Here is an example:

import re

code = """\
while (true){
    print("hello world")
}
"""

for line in code.splitlines():
    print(re.split(r"(\W+)", line))

You'll get:

['', '    ', 'while', ' (', 'true', '){', '']
['', '        ', 'print', '("', 'hello', ' ', 'world', '")', '']
['', '    }', '']
['', '    ', '']

With a filtering, you can drop empty string…

Or, if you need to match one-character symbols, you cantry:

for line in code.splitlines():
    tokens = [token for token in re.split(r"(\W)", line) if token.strip()]
    print(tokens)

You get:

['while', '(', 'true', ')', '{']
['print', '(', '"', 'hello', 'world', '"', ')']
['}']
[]

Answer 2

Try this:

import re

re1 = r'(.?)([(){}:,=+-]{1})(.?)'

lines = '''
while (true){
    print("hello world")
}
'''

words = []
for line in lines.split('\n'): #line holds array of the above lines
    cleanLine = re.sub(re1, '\g<1> \g<2> \g<3>', line)
    words += re.sub("[\s]", " ", cleanLine).split()}

print(words)
# ['while', '(', 'true', ')', '{', 'print', '(', '"hello', 'world"', ')', '}']