归档具有不同名称的每小时文件

Question

I have a log file which looks like this:

2018/10/08 12:15:04 David access denied
2018/10/08 12:15:05 David access denied
2018/10/08 12:15:05 David access granted
2018/10/08 13:15:14 Karel Jan access granted
2018/10/08 13:15:19 Lydia access denied
2018/10/08 13:15:20 Lydia access denied
2018/10/08 13:15:21 Lydia access granted
2018/10/08 14:15:26 Henk access denied
2018/10/08 14:15:26 Henk access denied
2018/10/08 14:15:27 Henk access denied

Script:

file="log.txt"
while read -r regel
do
        sort | awk '{file=$1 substr($2,1,2); gsub(/[^0-9]/,"",file) }
                {print > ("logfile_" file ".txt")}'
        zip logfile_20181008.zip logfile_20181008{00..23}.txt   
done < "$file"

This is what I got so far with help, also getting the errors below:

ArchiveerLog.sh: line 6: syntax error near unexpected token `('
ArchiveerLog.sh: line 6: `  {print > (prefix bestand".txt")}'

I have hourly logfiles, and want to zip them for each day so the zip would be called logfile_20181008.zip as above, is there a way to NOT hardcore this?

Answer 1

If you have an unsorted log-file with a log-key of this type, you are a bit in an uneasy position. Sortable date-time formats are of the form YYYY[c]MM[c]DD[c]hh[c]mm[c]ss[.sss] and are always represented in the same Time zone. The format you present is not directly sortable by means of a simple ascii ordering. As a simple example. The keys "01/01/2018.00:00:00" < "01/10/2018.00:00:00" < "10/10/1302.00:00:00".

Using the tool sort you can set up a complicated sorting structure:

sort -k1.7n,1.10 -k1.4n,1.5 -k1.1n,1.2 -k1.11 <logfile>

This will sort your file correctly. Now you can pipe this into awk to do the splitting by hour :

sort -k1.7n,1.10 -k1.4n,1.5 -k1.1n,1.2 -k1.11 <logfile> \
    | awk -v prefix="logfile_" '{file=substr($1,1,13); gsub(/[^0-9]/,"",file) }
                                {print > (prefix file".txt")}'

This will sort the file and move all lines to the files logfile_DDMMYYYYhh.txt

Update : After the question has been updated!

sort log.txt \
 | awk '{file=$1 substr($2,1,2); gsub(/[^0-9]/,"",file) }
        {print > ("bestand_" file ".txt")}'

Second update: so your entire script can now be written as:

#!/usr/bin/env bash

#######################################################
# THIS IS NOT TESTED BUT SHOULD BE UPDATED WHERE NEEDED
#######################################################

# This is your input
logfile="log.txt"

# create a temporary directory where to do all the work
tmpdir=$(mktemp -d)

# get the full path of logfile
logfile=$(readlink -f "$logfile")

cd "$tmpdir" || exit
sort "$logfile" | awk '{file=$1 substr($2,1,2); gsub(/[^0-9]/,"",file) }
                       {print > ("logfile_" file ".txt")}'

# Now you have a $tmpdir with lots of files
# perform the zipping
oldstring=""
newstring=""
for files in ./*; do
   #remove last 6 characters ("hh.txt")
   newstring="${files%[0-9][0-9].txt}"
   [[ "${newstring}" == "${oldstring}" ]] && continue
   zip "$(dirname $logfile)/${newstring}.zip" ${newstring}[0-9][0-9].txt
   oldstring="${newstring}"
done

# uncomment this part only if you are sure it works
# # cleanup
# cd $(dirname $logfile)
# rm -rf "$tmpdir"