Python: How to find all matches in a multiline string but not proceeded by particular word?

Question

I have SQL codes and I would like to extract the table name after the "insert" keyword.

Basically, I would like to extract using the following rules:

1. Contains the word "insert"
2. Followed by the word "into" which is optional
3. Exclude if the there's a "--" (which is single line comment in SQL) anywhere before the insert into(optional) keyword.
4. Exclude if insert into(optional) keyword is between "/*" and "*/" (which is multiline comment in SQL).
5. Get the next word (table_name) after insert into (optional) keyword

Example:

import re

lines = """begin insert into table_1 end
    begin insert table_2 end   
    select 1 --This is will not insert into table_3
    begin insert into
        table_4
    end
    /* this is a comment
    insert into table_5
    */
    insert into table_6
    """

p = re.compile( r'^((?!--).)*\binsert\b\s+(?:into\s*)?.*', flags=re.IGNORECASE | re.MULTILINE)
for m in re.finditer( p, lines ):
    line = lines[m.start(): m.end()].strip()

    starts_with_insert = re.findall('insert.*', line, flags=re.IGNORECASE|re.MULTILINE|re.DOTALL)
    print re.compile('insert\s+(?:into\s+)?', flags=re.IGNORECASE|re.MULTILINE|re.DOTALL).split(' '.join(starts_with_insert))[1].split()[0]

Actual Result:

table_1
table_2
table_4
table_5
table_6

Expected Result: table_5 should not be returned since it's between /* and */

table_1
table_2
table_4
table_6

Is there an elegant way to do this?

Thanks in advance.

EDIT : Thanks for your solutions. Is it possible to use purely regex without stripping lines from original text?

I would like to display the line number where table name can be found from the original string.

Updated code below:

import re

lines = """begin insert into table_1 end
    begin insert table_2 end   
    select 1 --This is will not insert into table_3
    begin insert into
        table_4
    end
    /* this is a comment
    insert into table_5
    */
    insert into table_6
    """

p = re.compile( r'^((?!--).)*\binsert\b\s+(?:into\s*)?.*', flags=re.IGNORECASE | re.MULTILINE)
for m in re.finditer( p, lines ):
    line = lines[m.start(): m.end()].strip()
    line_no = str(lines.count("\n", 0, m.end()) + 1).zfill(6)

    table_names = re.findall(r'(?:\binsert\s*(?:into\s*)?)(\S+)', line, flags=re.IGNORECASE|re.MULTILINE|re.DOTALL)
    print '[line number: ' + line_no + '] ' + '; '.join(table_names)

Tried using lookahead/lookbehind to exclude those between /* and */ but it's not producing my expected result.

Would appreciate your help. Thanks!

Answer 1

In 2 steps with re.sub() and re.findall() functions:

# removing single line/multiline comments
stripped_lines = re.sub(r'/\*[\s\S]+\*/\s*|.*--.*(?=\binsert).*\n?', '', lines, re.S | re.I)

# extracting table names preceded by `insert` statement 
tbl_names = re.findall(r'(?:\binsert\s*(?:into\s*)?)(\S+)', stripped_lines, re.I)
print(tbl_names)

The output:

['table_1', 'table_2', 'table_4', 'table_6']

Answer 2

import re
import string

lines = """begin insert into table_1 end
    begin insert table_2 end
    select 1 --This is will not insert into table_3
    begin insert into
        table_4
    end
    /* this is a comment
    insert into table_5
    */
    insert into table_6
    """

# remove all /* */ and -- comments
comments = re.compile('/\*(?:.*\n)+.*\*/|--.*?\n', flags=re.IGNORECASE | re.MULTILINE)
for comment in comments.findall(lines):
    lines = string.replace(lines, comment, '')

fullSet = re.compile('insert\s+(?:into\s+)*(\S+)', flags=re.IGNORECASE | re.MULTILINE)
print fullSet.findall(lines)

gives

['table_1', 'table_2', 'table_4', 'table_6']