使用linux过滤apache文件

Question

I'm trying to filter down a heap of apache log files to EXCLUDE all requests that have:

• The pattern /static/ (this is my images/js folder that I want to exclude)
• 10.xxx.xxx.xxx (where x is any number - I don't want internal requests included)
• Any response other than "GET / HTTP/1.1" 200 - only want successes

I have a folder containing multiple .gz files. Is there a way to run a linux command that will do the proper filtering and save the results in a file called apache_log.txt?

I'm really limited in my linux knowledge so will appreciate any help greatly!

Answer 1

For each file *.gz, uncompress and filter out unwanted static and local, and filter wanted "GET 200", and append this in result file.

for f in *.gz ; do zcat $f | grep -v '/static/' | grep -v '10\.[0-9]\+\.\.[0-9]\+\.[0-9]\+' | grep 'GET / HTTP/1.1" 200' >> apache_log.txt ; done

Or on multiple lines.

for f in *.gz
do
    zcat $f \
        | grep -v '/static/' \
        | grep -v '10\.[0-9]\+\.\.[0-9]\+\.[0-9]\+' \
        | grep 'GET / HTTP/1.1" 200' \
        >> apache_log.txt
done