锻炼的数据结构建议

Question

As a part of Data Structure course my teacher gave me an extra exercise which is a little bit more difficult and challenging. I`ve tried to find out the Data Structure I need to use for this problem and I dont have any idea, also I want to try to code it by myself out of the exercise to improve my python skills.

About the exercise: 1. I have a text file with logs which looks like that:

M, 1, 14/08/2019 11:39, 4, xxxx, username, “Initialization of the system, and loading
M, 1, 14/08/2019 11:40, 100, xxxx, username, “Open Connection”
M, 1, 14/08/2019 11:40, 100, xxxx, username, “Close Connection, and reboot”
S, 1, 14/08/2019 11:40, 6, xxxx, New User, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, xxxx, User logged in, User username logged in
M, 1, 14/08/2019 11:39, 4, xxxx, username, “Initialization of the system”
S, 1, 14/08/2019 11:40, 6, xxxx, New User, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, xxxx, User logged in, User username logged in

there are 2 types of logs, M is Master and S is Slave. I need a data structure which will be able to split each row and grab it into a specific column. ie M-1 columns will be :

M, 1, Datetime, Error Level, DeviceId, UserId, Message

but S-1 columns will be :

S, 1, Datetime, Error Level, DeviceId, Action, Message

Note : as you can see there is Action in S,1 but not UserId.

What I need to be able to do at the end is to enter in the command line the columns that I want to stdout and a condition (ie Error Level > 50).

What I tought about was Dictionary, but by this way I won't be able to support unlimited number of versions (if its possible, please explain me how).

Thanks!

Answer 1

I would probably use a namedtuple class from the collections package to hold each parsed item since it allows you to access each field by an index number and also by name. Moreover, new namedtuple classes can be dynamically created rather easily by passing a list of column names.

from collections import namedtuple

Master = namedtuple('Master', ['Type', 'N', 'Datetime', 'ErrorLevel', 'DeviceId', 'UserName', 'Message'])
Slave = namedtuple('Slave', ['Type', 'N', 'Datetime', 'ErrorLevel', 'DeviceId', 'Action', 'Message'])

n_cols = 7

logfileasstring = """
M, 1, 14/08/2019 11:39, 4, xxxx, username, “Initialization of the system, and loading
M, 1, 14/08/2019 11:40, 100, xxxx, username, “Open Connection”
M, 1, 14/08/2019 11:40, 100, xxxx, username, “Close Connection, and reboot”
S, 1, 14/08/2019 11:40, 6, xxxx, New User, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, xxxx, User logged in, User username logged in
M, 1, 14/08/2019 11:39, 4, xxxx, username, “Initialization of the system”
S, 1, 14/08/2019 11:40, 6, xxxx, New User, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, xxxx, User logged in, User username logged in"""


master_list = []
slave_list = []

for r in logfileasstring.splitlines(False):
    if not r:
        continue
    values = [value.strip() for value in r.split(',', n_cols - 1)]
    if r[0] == 'M':
        master_list.append(Master(*values))
    else:
        slave_list.append(Slave(*values))


print(master_list[0][6]) # by index
print(master_list[0].Message) # by column name if name known in advance
column_name = 'Message'
print(master_list[0].__getattribute__(column_name)) # by column name if name not known in advance

Run demo

Answer 2

does this help:

logfileasstring = """
M, 1, 14/08/2019 11:39, 4, xxxx, username, “Initialization of the system, and loading
M, 1, 14/08/2019 11:40, 100, xxxx, username, “Open Connection”
M, 1, 14/08/2019 11:40, 100, xxxx, username, “Close Connection, and reboot”
S, 1, 14/08/2019 11:40, 6, xxxx, New User, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, xxxx, User logged in, User username logged in
M, 1, 14/08/2019 11:39, 4, xxxx, username, “Initialization of the system”
S, 1, 14/08/2019 11:40, 6, New User, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, User logged in, User username logged in"""
listoflist = [[v.strip() for v in r.split(",", maxsplit=6)]
               for r in logfileasstring.splitlines(keepends=False) 
               if r]

grouped = {("M", "1"): [], ("S", "1"): []}
for row in listoflist:
    datasets_for = grouped[row[0], row[1]]
    datasets_for.append(row[2:])


# must be set by script
fields = [0, 1, 2]
for k in grouped:
    print(k, "::")
    for row in grouped[k]:
        print("  -", [row[f] for f in fields])