在Linux的日志文件中找到两行的日期之间的差异

Question

I have a file which has data in below format :

2016-08-26 14:43:00,840[http-nio-8014-exec-2]INFO
2016-08-26 14:43:00,877[http-nio-8014-exec-2]INFO
2016-08-26 14:43:01,130[http-nio-8014-exec-9]INFO
2016-08-26 14:43:01,171[http-nio-8014-exec-9]INFO
2016-08-26 14:43:01,484[http-nio-8014-exec-8]INFO
2016-08-26 14:43:01,523[http-nio-8014-exec-8]INFO
2016-08-26 14:43:02,091[http-nio-8014-exec-1]INFO
2016-08-26 14:43:02,132[http-nio-8014-exec-1]INFO

I want to calculate date difference between line 2 and 1 , line 4 and 3 and so on. and want output line below

Difference between line 2 and 1 is 37 ms
Difference between line 4 and 3 is 41 ms
Difference between line 6 and 5 is 39 ms

Need help in this.

Answer 1

try following

python sample.py sample.log

sameple.py

import fileinput
from datetime import datetime
import time, datetime, re

dt_pat = re.compile(r"(\d+)\-(\d+)-(\d+)\s+(\d+):(\d+):(\d+),(\d+)")
def mstime( tstr ):
    m = dt_pat.match( tstr )
    if m==None: return m
    v = [int(s) for s in m.groups()]
    return datetime.datetime(*v)

content = []
for line in fileinput.input():
    content.append(line.strip().split('[')[0])

for time in range(0,len(content)-1):
     diff = mstime( content[time+1])-mstime( content[time])
     print "{:f}".format(float(diff.total_seconds()))

fileinput.close()

Answer 2

If you wanted to stick to POSIX shell as your question is tagged, you could do something similar to the following:

#!/bin/sh

declare -i cnt=0
last=""
while read -r line; do
    ((cnt == 0)) && { ((cnt++)); last="$line"; continue; }

    tline="${line%%[*}"
    tline="${tline##*,}"
    dindex=$(expr index "$tline" [1-9])
    ((dindex > 0)) && tline=$(expr substr "$tline" "$dindex" 3)

    tlast="${last%%[*}"
    tlast="${tlast##*,}"
    dindex=$(expr index "$tlast" [1-9])
    ((dindex > 0)) && tlast=$(expr substr "$tlast" "$dindex" 3)

    tmdiff=$((tline - tlast))
    ((tmdiff < 0)) && ((tmdiff+=1000))

    printf "Difference between line %d and %d is %s ms\n" \
    "$((cnt + 1))" "$cnt" $tmdiff

    last="$line"
    ((cnt++))
done < "$1"

( note: above presumes difference between entries is less than 1 sec, and some versions of read don't support the -r option (just remove it if that is the case))

Example Use/Output

Just pass the log filename as the first argument to the script:

$ sh logtmdiff.sh dat/logtimes.txt
Difference between line 2 and 1 is 37 ms
Difference between line 3 and 2 is 253 ms
Difference between line 4 and 3 is 41 ms
Difference between line 5 and 4 is 313 ms
Difference between line 6 and 5 is 39 ms
Difference between line 7 and 6 is 568 ms
Difference between line 8 and 7 is 41 ms

Answer 3

Thanks to Ravichandra, The script worked for me by some minor change as i wanted the deference between line 2 & 1 and then 4 & 3 ...So the iterator index time has to be incremented by 2.

import fileinput
from datetime import datetime
import time, datetime, re

dt_pat = re.compile(r"(\d+)\-(\d+)-(\d+)\s+(\d+):(\d+):(\d+),(\d+)")
def mstime( tstr ):
    m = dt_pat.match( tstr )
    if m==None: return m
    v = [int(s) for s in m.groups()]
    return datetime.datetime(*v)

content = []
for line in fileinput.input():
    content.append(line.strip().split('[')[0])

for time in range(0,len(content)-1,2):
     diff = mstime( content[time+1])-mstime( content[time])
     print "{:f}".format(float(diff.total_seconds()))

fileinput.close()