无法使用Python解析目录中的多个文件

Question

I have 2 files in a directory. One directory contains start and end time of ETL jobs in morning and the other contains same data for the evening. I am trying to write a Python program to read the file and their contents and give an excel output which contains the file name, date, start time and end time.

My Code is written below:

path = r"path_name"
regex = '(.*?) - (.*?) - Starting entry (.*?)'
regex_1 = '(.*?) - (.*?) - Clear TMP table'
regex_2 = '(.*?) - (.*?) - Finished job'
for filename in glob.glob("*.log"):
    with open(filename, "r") as file:
        file_list = []
        table_list = []
        start_list = []
        end_list = []
        for line in file:
            line = line.replace('[','')
            line = line.replace(']','')
            line = line.replace('(','')
            line = line.replace(')','')
            for match in re.finditer(regex, line, re.S):
                match_text = match.group()
                print match_text
                searchfile = re.search(' - (.+?) - ', match_text)
                if searchfile:
                    filename = searchfile.group(1)
                    file_list.append(filename)
                    print(filename)
            for match in re.finditer(regex_1, line, re.S):
                match_text_1 = match.group()
                print match_text_1      
                searchtable = re.search(' - (.+?) - ', match_text_1)
                if searchtable:
                    tablename = searchtable.group(1)
                    table_list.append(tablename)
                    print(tablename)
                    starttime = match_text_1[0:19]
                    start_list.append(starttime)
                    print(starttime)
            for match in re.finditer(regex_2, line, re.S):
                match_text_2 = match.group()
                print match_text_2 
                endtime = match_text_2[0:19]
                end_list.append(endtime)
                print(endtime)

The issue here is that only one file is being read and written. I am not able to understand why that's happening. If I am printing the length of file_list, it contains 400 rows but ideally there should be 800 rows since I am parsing 2 files. Can someone pls help me with this?

Answer 1

Initialize file_list outside the loop and then use append to populate data.

ie

file_list = []
for filename in glob.glob('*.log'):
    if some_condition:
        file_list.append(filename)

In your case, the file_list gets initialized in every iteration and hence only half the data is present.