How to read log file for last n min in linux

Question

I have file in following format and I want read the file for last n no of minutes.

2019-09-22T04:00:03.052+0000: 774093.613: [GC (Allocation Failure)
Desired survivor size 47710208 bytes, new threshold 15 (max 15)
[PSYoungGen: 629228K->22591K(650752K)] 1676693K->1075010K(2049024K), 0.0139764 secs] [Times: user=0.05 sys=0.00, real=0.01 secs]

I want to read the log file for xn of minutes based on user requirement so that I can monitor it for last 30 min or 120 min based on user requirement.

I have tried below option to read the file but seems its not working as expected:

awk -F - -vDT="$(date --date="60 minutes ago" +"%Y-%m-%dT%H:%M:%S")" ' DT > $NF,$0' gc-2019-09-13-04-58.log.0.current

Also, in above command "60 minutes ago" option is there which I tried to pass as a variable like v1=30 , date --date="$v1 minutes ago" , this one also not working?

Please suggest how to read this file for last x no of minutes?

Answer 1

Here is one for GNU awk ( time functions and gensub() ). First the test data, two lines of your data with year changed in the first one:

2018-09-22T04:00:03.052+0000: 774093.613: [GC (Allocation Failure)
Desired survivor size 47710208 bytes, new threshold 15 (max 15)
[PSYoungGen: 629228K->22591K(650752K)] 1676693K->1075010K(2049024K), 0.0139764 secs] [Times: user=0.05 sys=0.00, real=0.01 secs]
2019-09-22T04:00:03.052+0000: 774093.613: [GC (Allocation Failure)
Desired survivor size 47710208 bytes, new threshold 15 (max 15)
[PSYoungGen: 629228K->22591K(650752K)] 1676693K->1075010K(2049024K), 0.0139764 secs] [Times: user=0.05 sys=0.00, real=0.01 secs]

and the awk program, to which the data is fed backwards using tac :

$ tac file | gawk '
BEGIN {
    threshold = systime()-10*60*60 # time threshold is set to 10 hrs
    # threshold = systime()-mins*60# uncomment and replace with above 
}                                  # for command line switch
{
    if(match($1,/^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}/)) {
        if( mktime( gensub(/[-T:]/," ","g",substr($1,RSTART,RLENGTH))) < threshold)
            exit                   # exit once first beyond threshold time is found
        print $0 b                 # output current record and the buffer
        b=""                       # reset buffer
    } else                         # for non-time starting records:
        b=ORS $0 b                 # buffer them
}'

You could write the program code between the ' s to a file, say program.awk and run it with tac file | gawk -f program.awk tac file | gawk -f program.awk and furthemore add a command line switch by uncommenting the marked line in the BEGIN section and running with:

$ gawk -v mins=10 -f program.awk <(tac file)

Answer 2

Get the last N lines of a log file. The most important command is "tail". ... Get new lines from a file continuously. To get all newly added lines from a log file in realtime on the shell, use the command: tail -f /var/log/mail.log. ... Get the result line by line. ... Search in a log file. ... View the whole content of a file.