Script Shell: grep specific lines - extract specific strings - put them on a file

Question

From the appache2 accesslog.log file I am trying to grep the lines where there is this string : "GET /kiosk/kioskconf.txt HTTP/1.1"

For this I am using the command :

grep "GET /kiosk/kioskconf.txt HTTP/1.1"

Which works fine. eg:

19x.25x.22x.5x - - [27/Apr/2015:14:15:50 +0200] "GET /kiosk/kioskconf.txt HTTP/1.1" 

There is multiple lines like that and i wana extract to a file just the IP in the begining of each line. also i don't want to have same Ip's in the file. i use this to extract the ip's but it's not complete.

sed -n 's/.*194\([^ ]*\).*/\1/p'

which displays:

.25x.22x.5x

but i wont the whole ip's and just one instance for each different ip's and put them on a file.

Can someone help me to sort this please?

Answer 1

Sounds like you should be using awk:

awk '/GET \/kiosk\/kioskconf\.txt HTTP\/1\.1/ && !seen[$1]++ { print $1 }' file

This prints the first field $1 whenever the pattern is matched but the IP address is not already in the array seen . It also increments the value of seen[$1] so next time the same IP occurs, the second part of the condition will be false and the line will not be printed.

/ and . have special meaning in the regular expression pattern, so they must be escaped.

Answer 2

You can use the sort | uniq sort | uniq commands to get unique values.

Answer 3

You could try

grep "GET /kiosk/kioskconf.txt HTTP/1.1" | cut -f1 | sort -u

This will just strip all but the first, space seperated field in your grep output (ie the IP addresses) and then sort them, removing duplicates.