在日志文件中查找每天的 min() 和 max()

Question

New to Python, but I am stumped.

I have a text log file that starts each entry with a timestamp. So :

03/17/2020 01:38:20 PM
03/18/2020 09:21:28 AM

I want to go through and create a two dimensional list that has one entry for each day as well as the earliest and latest timestamp found. For instance, the list would contain [3/17/2020, 09:00:00 AM, 01:26:16 PM], [4/28/2020, 10:14:00 AM, 03:16:16 PM], with additional entries for each day.

Here is what I have so far (I blew away a previous attempt)

    lActDays = []
lActDayTimes = []
for item in lUAData:
    # Find the first space in the Time column.
    ispaceindx = item[0].find(' ')
    # Use the space as a delimiter, print everything before that - should be the date.
    sRecDay = item[0][0:ispaceindx]
    # Use the space as a delimiter, print everything after - should be the time.
    sRecTime = item[0][ispaceindx:].strip()
    if not sRecDay in lActDays:
        lActDays.append([sRecDay, [sRecTime]])

When I run this, it keeps appending [sRecDay, [sRecTime]] each time the for loop runs. Its like the 'if not' condition isn't being run. However, if I change the last line to lActDays.append(sRecDay), it works fine. I get a list of unique days (but without a time)

Answer 1

You want to sort dates, you'll want to take advantage of the special properties of datetime objects that allow them to be sorted. Since you're reading in what looks to be a predictable format, you can also take advantage of datetime parsing and output formatting :

from datetime import datetime

entries = ['06/05/2020 09:21:00 AM log file line 1 text',
           '06/15/2020 10:59:59 PM log file line 2 text',
           '06/25/2020 04:12:58 AM log file line 3 text',
           '06/05/2020 07:24:11 AM log file line 4 text',
           '06/15/2020 08:18:56 PM log file line 5 text',
           '06/25/2020 03:46:00 AM log file line 6 text',
           '06/05/2020 09:40:57 PM log file line 7 text',
           '06/15/2020 08:50:35 PM log file line 8 text',
           '06/25/2020 09:30:45 PM log file line 9 text',
           '06/05/2020 01:40:14 AM log file line 10 text']

fp = 'dummyfile.txt'
with open(fp, 'w') as dfile:
    dfile.write("\n".join(entries))


def get_daily_first_last(file_path):
    fmt = '%m/%d/%Y %H:%M:%S %p'
    with open(file_path, 'r') as infile:
        data = {}
        for line in infile:
            dt, txt = datetime.strptime(line[:22], fmt), line.strip()
            day = dt.date().isoformat()
            if day in data.keys():
                data[day].append((dt, txt))
            else:
                data[day] = [(dt, txt)]

    for k, v in data.items():
        v = sorted(v)
        print(f"Day: {k}\nFirst entry: {v[0][1]}\nLast entry: {v[-1][1]}")


if __name__ == "__main__":
    get_daily_first_last(fp)

You should get the output:

Day: 2020-06-05
First entry: 06/05/2020 01:40:14 AM log file line 10 text
Last entry: 06/05/2020 09:40:57 PM log file line 7 text
Day: 2020-06-15
First entry: 06/15/2020 08:18:56 PM log file line 5 text
Last entry: 06/15/2020 10:59:59 PM log file line 2 text
Day: 2020-06-25
First entry: 06/25/2020 03:46:00 AM log file line 6 text
Last entry: 06/25/2020 09:30:45 PM log file line 9 text

The "entries" list and "dummyfile.txt" is just an example to show it works. You asked for a list, but I really think you want a dictionary for this problem, so you can group data as you're parsing the file. I'm storing the entire line to the second item of a tuple in the data dictionary, so I can just print that out after it's sorted. The first item int the tuple is the datetime object that supports comparisons (eg sorting). The v = sorted(v) line returns a list that is sorted by the first item in each tuple (the datetime object).

Answer 2

You have a bunch of questions in one. You really should break things down into separate tasks. That way, when you ask a question, it's one issue that you're working on, and you have code and data to demonstrate the issue, and your progress so far.

I put together this small demo based on the data you gave. Normally, you would want a regex to parse the date and separate it from the rest of the log data (see Extracting a date from a log file? )

But since your date is a fixed length, here's a super quick and dirty way to parse it, assuming it's date-space-text:

import datetime
import re
from collections import defaultdict
from pprint import pprint

logfile = [
    '03/17/2020 01:38:20 PM stuff goes here',
    '03/18/2020 08:21:28 AM earlier',
    '03/18/2020 09:21:28 AM more stuff in this line',
    '03/18/2020 11:21:28 AM later',
]

print ('parsing debug prints:')
mydata = defaultdict(list)
for line in logfile:
    timestamp, message = line[:22], line[23:]
    dt = datetime.datetime.strptime(timestamp, '%m/%d/%Y %I:%M:%S %p')
    date_string = dt.strftime('%m/%d/%Y')
    print (date_string)
    print (message)
    mydata[date_string].append((dt, message))
print()

print ('The full data structure:')
pprint(mydata)
print()

There's nothing special about finding the min/max items, assuming those items are a type that supports comparison. That is why I have put your timestamps into a datetime.

day = '03/18/2020'
list_of_records_from_day = mydata[day]
list_of_datetime_objects = [r[0] for r in list_of_records_from_day]
print ('The earliest timestamp on', day, 'is', min(list_of_datetime_objects))