Python-从日志文件读取10分钟

Question

I need some tool to read latest 10 minutes entry in my log file, and if some words are logged then print some text.

log file:

23.07.2014  09:22:11   INFO Logging.LogEvent   0  Failed login test@test.com
23.07.2014  09:29:02   INFO Logging.LogEvent   0  login test@test.com
23.07.2014  09:31:55   INFO Logging.LogEvent   0  login test@test.com
23.07.2014  09:44:14   INFO Logging.LogEvent   0  Failed login test@test.com

if during last 10min some entry = Failed -print ALARM.

All what i did is find 'Failed' match but i have no idea how to check last 10min in my log file ;/ -any idea??

from sys import argv
from datetime import datetime, timedelta

with open('log_test.log', 'r') as f:
    for line in f:
        try:
            e = line.index("Failed")
        except:
            pass
        else:
            print(line)

Answer 1

Your format %d.%m.%Y is worse than %Y:%m:%d which can be used in string comparison.

We also do not know if log is big and if it is sorted. If it is not sorted (it is common for multithreaded applications) you will have to analyze each line and convert it into datetime:

def get_dt_from_line(s):
    return datetime.datetime.strptime(s[:20], '%d.%m.%Y  %H:%M:%S')

Then use it as filter (for small files):

    MAX_CHECK_TIMEDELTA = datetime.timedelta(minutes=10)
    LOG_START_ANALYZE_DATETIME = (datetime.datetime.today() - MAX_CHECK_TIMEDELTA)
    lines = [s for s in TXT.split('\n') if 'Failed' in s and get_dt_from_line(s) >= LOG_START_ANALYZE_DATETIME]
    print('\n'.join(lines))

For big files you can read file line by line.

If your log file is just for one day you can use string comparison instead of datetime comparison:

LOG_START_ANALYZE_DATETIME = (datetime.datetime.today() - datetime.timedelta(minutes=10)).strftime('%d.%m.%Y  %H:%M:%S')
lines = [s for s in TXT.split('\n') if 'Failed' in s and s >= LOG_START_ANALYZE_DATETIME]

Answer 2

If I were you, I would lookup line by line, get the timestamp of the first line and then iterate until the difference between the first date and the current one is more than 10 minutes, while counting occurences of the word "Failed".

I think that you'll sort something out with splitting your line following spaces. But be careful as if someday, your log format changes, your script is likely not gonna be working too.