awk正则表达式提取网址的一部分

Question

I am very new to awk. I have multiple files containing lines similar to

xyz msg=(1448783938.658:149777):   uid=505   comm="abc.py"   exe="/install/python/bin
abc msg=(1448783938.658:149777):   uid=506   comm="abc.py"   exe="/install/bio/toolx/bin
abc msg=(1448783938.658:149777):   uid=505   comm="abc.py"   exe="/install/bio/tooly/bin

I need two outputs. a. One file containing just the uid and exe column (just after /install/) from across all the files eg

505 python
506 bio
505 bio

I can print the exe with

awk -F '/' '{ print $3}'

but unsure how to print the uid with it.

2. One file containing uid and exe column containing just string following /bio/ eg

   506 toolx

   505 tooly

Any help appreciated.

Answer 1

You can use the following awk command:

awk -F'[[:space:]="/]+' '{print $5, $10}' file

I'm using a set of delimiters. This makes it simple to access the values of interest. However, it works only if the path contains no <space> , = , " and / .

Btw, also sed can be used for that, this would work regardless of whatever kind of characters are in the path since no delimiter is used:

sed -r 's~.*uid=([^[:space:]]+).*exe="/install/([^/]+).*~\1 \2~' file

Answer 2

similar awk solution

$ awk -F" +|[=/]" '{print $5,$11}' bio
505 python
506 bio
505 bio

Answer 3

I would keep it simple and use the default field delimiter -- then use sub or split to clean up each field for printing. Here is the split solution.

awk '{ split($3, uid, "="); split($5, exe, "/"); print uid[2], exe[3] }'

Here's how this was developed:

$ echo 'xyz msg=(1448783938.658:149777):   uid=505   comm="abc.py"   exe="/install/python/bin' | awk '{ print $3, $5 }'
uid=505 exe="/install/python/bin
$ echo 'xyz msg=(1448783938.658:149777):   uid=505   comm="abc.py"   exe="/install/python/bin' | awk '{ split($3, uid, "="); print uid[2], $5 }'
505 exe="/install/python/bin
$ echo 'xyz msg=(1448783938.658:149777):   uid=505   comm="abc.py"   exe="/install/python/bin' | awk '{ split($3, uid, "="); split($5, exe, "/"); print uid[2], exe[3] }'
505 python

I tried a sub based solution first, but that turned out to be both longer and more cryptic than the split based solution -- the split solution seemed more straightforward. (In the case a sub based solution is warranted, perhaps sed would be a better candidate language anyway.)

One thing that should be added is some filter to make sure we only process valid lines, which could be as simple as the following:

awk '$3 ~ /uid=/ && $5 ~ /exe="\/install\// { split($3, uid, "="); split($5, exe, "/"); print uid[2], exe[3] }'

One other thing... If uid and exe fields move around in your file from column to column, you will have to hunt for them using a for loop... which is long enough to turn into a script file like the following:

#! /usr/bin/awk -f
{
        u=0
        e=0
        for (i=1; i<=NF; i++) {
                if ($i ~ /uid=/)
                        u=i
                else if ($i ~ /exe="\/install\//)
                        e=i
                if (u && e)
                        break
        }
        if (!u || !e)
                next
        split($u, uid,"=")
        split($e, exe, "/")
        print uid[2], exe[3]
}

In this case, the leading pattern that checks for validity that we put in the immediately preceding example are embedded in the for loop.