通过tail格式化和漂亮的打印日志

Question

I have this log file that I check on a frequent basis and because of the format of it, it's quite easier to read when pretty printed. I'd like to do so in a tail.

Logs in the file like:

2019-07-04T09:53:04-07:00   some.package.placeholder.stderr {"log": "The content", "foo": "bar", "baz": "blah"}
2019-07-04T10:15:37-07:00   some.package.placeholder.stderr {"log": "I'm actually", "foo": "bar", "baz": "blah"}
2019-07-04T10:15:37-07:00   some.package.placeholder.stderr {"log": "Interested on", "foo": "bar", "baz": "blah"}

And I want to do something similar to

tail -f myLogFile | grep [...?...] | jq '.log'

So when tailing I get:

The content
I'm actually
Interested on

Or even:

2019-07-04T09:53:04-07:00   The content
2019-07-04T10:15:37-07:00   I'm actually
2019-07-04T10:15:37-07:00   Interested on

Answer 1

With GNU grep for -o :

$ tail file | grep -o '{[^}]*}' | jq -r '.log'
The content
I'm actually
Interested on

With any awk:

$ tail file | awk 'sub(/.*{/,"{")' | jq -r '.log'
The content
I'm actually
Interested on

$ tail file | awk '{d=$1} sub(/.*{/,""){$0="{\"date\": \""d"\", " $0} 1' | jq -r '.date + " " + .log'
2019-07-04T09:53:04-07:00 The content
2019-07-04T10:15:37-07:00 I'm actually
2019-07-04T10:15:37-07:00 Interested on

That last one works by merging the date field from the input into the json so then jq can just select and print it with the log field.

Answer 2

If the log lines are tab delimited, you can read the lines in raw and split on tabs. Which you could then parse the json and filter to your hearts content, and recombine as necessary.

$ tail -f myLogFile | jq -Rr 'split("\t") | [.[0], (.[2] | fromjson.log)] | join("\t")'
2019-07-04T09:53:04-07:00   The content
2019-07-04T10:15:37-07:00   I'm actually
2019-07-04T10:15:37-07:00   Interested on

Answer 3

Here's a thing I use that can be used in a pipe and with file args:

cat /usr/local/bin/j2t
#!/bin/bash

function usage {
  cat <<EOF
Usage:
        $0 <json filename>
    or
        tail -F <json filename> | $0
EOF
}

if (($# == 0)); then
    {
        sed "s/@\(timestamp\)/\1/" | jq -r '[.timestamp, .pri.facility, .pri.severity, .message] | join("\t")'
    } < /dev/stdin

else
    if [ -r "$1" ] ; then
        sed "s/@\(timestamp\)/\1/" $1 | jq -r '[.timestamp, .pri.facility, .pri.severity, .message] | join("\t")'
    else
        help
    fi
fi

eg: (if your daemon.log is json)

j2t /var/log/daemon.log
level: 63, builder: awillia2)
2021-08-14T00:00:06.820642+00:00        daemon  INFO     Starting Run Clamscan...
2021-08-14T00:00:06.846405+00:00        daemon  INFO     Started Run Clamscan.

Should probably reformat the time, it's a bit long.