重新格式化Python中的文本

Question

I am new to python and having great difficulty parsing through a log file. Can you please help me understand how I can accomplish the below in the most Pythonic way.

----- Log Entry 5 -----
Time       : 2016-07-12 09:00:00
Animal     : Brown Bear
Bird       : White Owl
Fish       : Salmon


----- Log Entry 6 -----
Time       : 2016-07-12 09:00:00
Animal     : Brown Bear
Bird       : Parrot
Fish       : Tuna


----- Log Entry 7 -----
Time       : 2016-07-12 09:00:00
Animal     : Lion
Bird       : White Owl
Fish       : Sword Fish


----- Log Entry 8 -----
Time       : 2016-07-12 09:15:00
Animal     : Lion
Bird       : White Owl
Fish       : Sword Fish

Desired Output 1: I would like to reformat the log to look like the below:

Time: 2016-07-12 09:00:00 Animal: Brown Bear  Bird: White Owl  Fish : Salmon
Time: 2016-07-12 09:00:00 Animal: Brown Bear  Bird: Parrot     Fish : Tuna
Time: 2016-07-12 09:00:00 Animal: Lion        Bird: White Owl  Fish : Sword Fish
Time: 2016-07-12 09:15:00 Animal: Lion        Bird: White Owl  Fish : Sword Fish

Desired Output 2: Then I would like to have the ability to query a time stamp and get a summary of counts:

Time: 2016-07-12 09:00:00
Name:       Count:
Brown Bear  2
Lion        1
White Owl   2
Parrot      1
Salmon      1
Tuna        1
Sword Fish  1

Time: 2016-07-12 09:15:00
Name:       Count:
Lion        1
White Owl   1
Sword Fish  1

My Code So Far:

import os, sys, time, re, collections, subprocess

show_cmd = 'cat question |  egrep -v \'^$|=|Log\' | awk \'ORS=NR%4?FS:RS\' | grep Time'
log = (subprocess.check_output(show_cmd, shell=True).decode('utf-8'))

def time_field():
    logRegex = re.compile(r'Time\s*:.*\d\d\d-\d\d-\d\d\s\d\d:\d\d')
    log_parsed = (logRegex.findall(log))
    a = (str(log_parsed).replace('  ', ''))
    a = ((' ' + a[1:-1]).split(','))
    for i in a:
        print(i)

time_field()

Answer 1

There are a lot of ways to do this. Personally I would avoid using regex for this because it probably won't be more efficient and the expression becomes cumbersome and inflexible. Here is something I came up with:

class Entry:
    def __init__(self):
        self.time = None
        self.animal = None
        self.bird = None
        self.fish = None

    def __repr__(self):
        fmt = "{0} {1} {2} {3}".format(
            "Time: {time: <{width}}",
            "Animal: {animal: <{width}}",
            "Bird: {bird: <{width}}",
            "Fish: {fish: <{width}}")
        return fmt.format(
            time=self.time, animal=self.animal,
            bird=self.bird, fish=self.fish,
            width=12)

    def __radd__(self, other):
            return self.__add__(other)

    def __add__(self, other):
        if type(other) == dict:
            for i in [self.animal, self.bird, self.fish]:
                if i in other: other[i] += 1
                else: other[i] = 1
            return other
        elif type(other) == Entry:
            return self.__add__({}) + other
        else:
            return self.__add__({})

def parse_log(path):
    def extract(line):
        start = line.find(':') + 1
        return line[start:].strip()

    entries = []
    entry = None
    with open(path, 'r') as f:
        for line in f.readlines():
            if line.startswith('-----'):
                if entry: entries.append(entry)
                entry = Entry()
            elif line.startswith('Time'):
                entry.time = extract(line)
            elif line.startswith('Animal'):
                entry.animal = extract(line)
            elif line.startswith('Bird'):
                entry.bird = extract(line)
            elif line.startswith('Fish'):
                entry.fish = extract(line)

        if entry: entries.append(entry)

    return entries


def print_output_1(entries):
    for entry in entries:
        print entry

def print_output_2(entries, time):
    animals = sum([e for e in entries if e.time == time])

    print "Time: {0}".format(time)
    print "Name:        Count:"
    for animal, count in animals.items():
        print "{animal: <{width}} {count}".format(
                animal=animal, count=count, width=12)


logPath = 'log.log'
time = '2016-07-12 09:15:00'
entries = parse_log(logPath)

print_output_1(entries)
print ""
print_output_2(entries, time)

The output (given that log.log matches the input you gave) is:

Time: 2016-07-12 09:00:00 Animal: Brown Bear   Bird: White Owl    Fish: Salmon
Time: 2016-07-12 09:00:00 Animal: Brown Bear   Bird: Parrot       Fish: Tuna
Time: 2016-07-12 09:00:00 Animal: Lion         Bird: White Owl    Fish: Sword Fish
Time: 2016-07-12 09:15:00 Animal: Lion         Bird: White Owl    Fish: Sword Fish

Time: 2016-07-12 09:15:00
Name:        Count:
White Owl    1
Sword Fish   1
Lion         1

The way this code works is to use object oriented programming to our advantage in order to simplify the tasks we need to do: store log entries, represent log entries in a specific format, and combine log entries according to a specific property.

First, note that the Entry object and its properties ( self.time , self.animal , self.bird , self.fish ) represents an entry in the log. Assuming that the information stored in its properties is correct, a method can be created to represent that information as a formatted string. The method __repr__() is called when python wants the string representation of an object, so it seemed like a good place to put this code. There is heavy use of the format function in this method, but it should be clear how it works after browsing the python documentation on format .

A method for combining these entry objects is needed in order to get the second output you specified. This can be done many ways and the way I did it is not necessarily the best. I used the __radd__() and __add__() methods which are called when the + operator is used on an object. By doing this, the code entry1 + entry2 or sum([entry1, entry2]) can be used to get the sum of the animals in both entries. The Entry class can not be used to store the result of the sum, however, because it cannot contain arbitrary information. Instead, I chose to use a dict object to be the result of summing two Entry objects. In order to sum more than two Entry objects, Entry must also be able to sum with a dict object because Entry + Entry + Entry results in dict + Entry .

The __add__() function checks if the object it is being added to is a dict object. If this is the case, it checks if each of the animals in the entry exist in the dict already. If not, it will add the animal as a key. Otherwise, it will increment the value of that key. __radd__() is similar to __add__() except that it is used in some special circumstances. See the python documentation for more information.

For the case where the object is an Entry , code could have been written to gather all of the animals from each Entry object and create a dict from that information, but since there is already code to add an Entry with a dict it is easier to first add one object to an empty dict and then add the resulting dict with the other Entry object.

For all other objects, the Entry will simply return the dict description of itself, or itself added with an empty dict .

Now all of the tools exist to accomplish the goals listed earlier. To get a string representation of an Entry that matches desired output 1, all that is needed is print entry or strrepr = str(entry) . To get desired output 2, a little more work is involved, but it is simply summing all entries that have the same self.time property and then displaying the resulting dict.

The last part of the code not covered is the parsing of the log to create a list of Entry objects. The code simply walks line by line through the log and populates an Entry with the information. I feel like this is pretty straightforward, but you can feel free to ask questions if it does not make sense.