在字符串/时间键之间提取并重新格式化文本

Question

I am having problem with extracting text between two strings.

I have log file like this (example data):

2018-12-31 09:49:24 addData [data=data]</br>
2018-12-31 09:49:25 publishData .......................
2018-12-31 09:49:26 createDoc [xml=
<mail>
    <recipent>doctor who</recipent>
</mail>]
<attempt>1</attempt>]
2018-12-31 09:49:26 createDoc [xml=
<clientHash>hash</clientHash>
<content>context</content>]
2018-12-31 09:51:27 exampleService [count=1]

My code: perl -ne 'print if (/09:40/ .. /09:50/)' server.log | sed -n '/createDoc/,/]/p' perl -ne 'print if (/09:40/ .. /09:50/)' server.log | sed -n '/createDoc/,/]/p'

My output is:

2018-12-31 09:49:26 createDoc [xml=<mail><recipent>doctor who</recipent>
</mail>]
<attempt>1</attempt>]
2018-12-31 09:49:26 createDoc [xml=
<clientHash>hash</clientHash>
<content>context</content>]

but I want to have only xml like this:

<element>
<mail><recipent>doctor who</recipent>
</mail>
<attempt>1</attempt>
</element>
<element>
<mail><recipent>doctor who</recipent>
</mail>
<clientHash>hash</clientHash>
<content>context</content>
</element>

Answer 1

I would use Awk for this. If you have GNU Awk, you can even parse the time stamps easily.

awk -v start=$(date -d "09:40" +%s) \
    -v end=$(date -d "09:50" +%s) '           
    /^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2} / {
        if ($0 ~ / createDoc \[xml=/) {
            split($1, ymd, /-/)
            split($2, hms, /:/)
            when = mktime(ymd[1] " " ymd[2] " " ymd[3] " " hms[1] " " hms[2] " " hms[3])
            p = (when >= start && when <= end)
            if (p) $0 = substr($0, 36)
        }
        else p = 0
    }
    p { sub(/\]$/, ""); print }' file.log

This is somewhat Linux-centric -- in addition to GNU Awk (for the mktime function) the date syntax is specific to GNU date . (On OSX try date -j %H:%M:%S 09:40:00 +%s .)

Answer 2

Let's say we have tmp.log which something like below.

2018-12-31 09:49:24 addData [data=data]</br>
2018-12-31 09:49:25 publishData .......................
2018-12-31 09:49:26 createDoc [xml=<mail><recipent>doctor who</recipent></mail>]<attempt>1</attempt>]
2018-12-31 09:49:26 createDoc [xml=<clientHash>hash</clientHash><content>context</content>]
2018-12-31 09:51:27 exampleService [count=1]

We can combine some basic commands to get desired output.

• Extract xml contained lines from whole log file.
• Split string according to '[' character and get second element.

cat tmp.log | grep xml | awk 'BEGIN { FS = "[" } ; { print $2 }'

This will produce something like that:

xml=<mail><recipent>doctor who</recipent></mail><attempt>1</attempt>]
xml=<clientHash>hash</clientHash><content>context</content>]

If you also want to get rid of from last character which is ' ] '. Add one more awk too.

cat tmp.log | grep xml | awk  'BEGIN { FS = "[" } ; { print $2 }'| awk  'BEGIN { FS = "]" } ; { print $1 }'

I know it's not the coolest way to do that, at least it's easy to understand and works.