Python: parse file line by line with different start and end marks

Question

I have a log file like this one with different start and end marks:

#Wiliam
#Arthur
#Jackie
high;
10 11 11;
#Jim
#Jill
#Catherine
#Abby
low;
girl;
10 11 11 11;
#Ablett
#Adelina
none;
5,8;

I need to parse it line by line to get the result as below:

[
  ['#Wiliam','#Arthur','#Jackie','high;','10 11 11;'],
  ['#Jim','#Jill','#Catherine','#Abby','low;','girl;','10 11 11 11;'],
  ['#Ablett','#Adelina','none;','5,8;']
]

Is there a solution?

Answer 1

It's understood that each sublist starts with # and ends with ; . This is exactly what this Pythonic generator implementation uses:

def read_lists():
    with open('data') as file:
        sublist = []
        previous_line = ''
        for line in file:
            line = line.strip()
            if line.startswith('#') and previous_line.endswith(';'):
                yield sublist
                sublist = []
            sublist.append(line)
            previous_line = line
        yield sublist

for sublist in read_lists():
    print(sublist)

['#Wiliam', '#Arthur', '#Jackie', 'high;', '10 11 11;']
['#Jim', '#Jill', '#Catherine', '#Abby', 'low;', 'girl;', '10 11 11 11;']
['#Ablett', '#Adelina', 'none;', '5,8;']

Answer 2

In order to parse the file – you need to find the pattern , which will lead you to successful data collection.

From your example, – I can see that you stop appending items in sublist when you read string with integers and semicolon. I'd try to do it this way:

import ast
result = []

with open(f,'rb') as fl:
    sublist = []
    for line in fl:            
        line = line.strip()
        sublist.append(line)
        if type(ast.literal_eval(line[0])) is int and line[-1] == ';':
            result.append(sublist)
            sublist = []

Answer 3

Here's my implementation. Not entirely sure what the is_terminator() logic should look like.

def is_terminator(tokens):
    """
    Return True if tokens is a terminator.

    """
    is_token_terminator = False    
    tokens = tokens.split()
    if len(tokens) > 0:
        token = tokens[-1]
        if token.endswith(";"):
            try:
                int(token[:-1])
            except ValueError:                
                pass # not an int.. and so not a terminator?
            else:
                 is_token_terminator = True
    return is_token_terminator


sublist = []
result = [sublist, ]
f = file("input.txt", "r")
for tokens in f.readlines():

    sublist.append(tokens)        

    if is_terminator(tokens):
        sublist = []
        result.append(sublist)

print result

Answer 4

This appends the lines to a sub-list until it reaches a point where the previous appended line ends with a semicolon but the current line does not. At that point, it creates a new sub-list and continues.

lst = [[]]

try:
    with open("log.log", "r") as f:
        i = 0 # Index of the sub-list

        for line in f:
            line = line.strip()

            if line[-1:] != ";" and lst[i] and lst[i][-1][-1:] == ";":
                i += 1 # Increment the sub-list index.
                lst.append([]) # Append a new empty sub-list.

            lst[i].append(line)
except FileNotFoundError:
    print("File does not exist.")

print(lst)