date：bash上OSX 10上的非法时间格式，但是该命令有效

Question

I'm writing a bash script that reads and parses data from log lines, and it's trying to convert the format. The script is long, but I'll focus on the error:

if [[ $line == *"Marked: Malicious, To:"* ]]; then

QUEUE_ID=`awk ' {print $13}'  <<< $line | sed 's/,//g'` # QUEUE ID... index of the array

F_TIME["X$QUEUE_ID"]=`awk '{print $3}'  <<< $line ` # Hour
F_DAY["X$QUEUE_ID"]=`awk '{print $2}'  <<< $line ` # Day
F_MONTH["X$QUEUE_ID"]=`awk '{print $1}'  <<< $line ` # Month

#Procesing and obtaining the diff

F_FULLTIME["X$QUEUE_ID"]="${F_DAY["X$QUEUE_ID"]}-${F_MONTH["X$QUEUE_ID"]}-$year-${F_TIME["X$QUEUE_ID"]}" # creating the time in desired format

s=`date -j -f "%d-%b-%Y-%H:%M:%S"  "${F_FULLTIME[X$QUEUE_ID]}"   +%s` # ERROR LINE!! Doesn't work!

#s=`date -j -f "%d-%b-%Y-%H:%M:%S"  "2-Mar-2016-22:24:33"   +%s` # Test line

echo ".....${F_FULLTIME["X$QUEUE_ID"]}....2-Mar-2016-22:24:33...."

echo  "time $s ...\n";

fi

Test process: I comment the test line (marked on the script), and I try to do conversion to variable s. This is the output for each key:

Failed conversion of ``2-Mar-2016-20:30:03'' using format ``%d-%b-%Y-%H:%M:%S''
date: illegal time format
usage: date [-jnu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
        [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]

.....2-Mar-2016-20:30:03....2-Mar-2016-22:24:33....

Date is not taking the format. And as you can see, desired format and the format I have on the variable are pretty much same.

Test #2: I comment the error line, and I remove comment the test line. This is the result:

.....2-Mar-2016-22:06:10....2-Mar-2016-22:24:33....
time 1456953873 ...

Works perfectly. And of course text is aparently same as my variable on the first try.

Answer 1

The code looks OK, although overly complex. Try this simplified version, which should be a little easier to debug. We're going to work with individual variables as much as possible, rather than array entries.

if [[ $line == *"Marked: Malicious, To:"* ]]; then

    # run `awk` once
    fields=$(awk '{print $1, $2, $3, $13}' <<< "$line")
    read f_month f_day f_time queue_id <<< "$fields"
    queue_id=${queue_id//,/}

    # Is the X really necessary?

    # F_TIME["X$queue_id"]=$f_time
    # F_DAY["X$queue_id"]=$f_day
    # F_MONTH["X$queue_id"]=$f_month

    #Procesing and obtaining the diff
    f_fulltime="$f_day-$f-month-$year-$f_time"
    # F_FULLTIME["X$queue_id"]=$f_fulltime

    s=$(date -j -f "%d-%b-%Y-%H:%M:%S" "$f_fulltime" )    
fi

Some questions to ask:

1. What are the exact values of queue_id , f_time , f_day , and f_month ? Use, eg, printf '%s' "$queue_id" | hexdump -C printf '%s' "$queue_id" | hexdump -C to see if there are any hidden characters from $line that are confusing date .

2. Can you reproduce the problem by hardcoding the values of the pieces instead of just f_fulltime itself?

3. Can you reproduce the problem by hard-coding the value of line just prior to the call to awk ?

Answer 2

Well.. I've clean the script.., Remove extra variables, declared arrays, and I found where the error comes.. but not sure why. The error comes on the variable $year.

This is defined as:

year=`date +"%Y"`

Then.. when I tried this conversion:

s_fulltime["X$queue_id"]="$day-$month-$year-$time" >> date: illegal time format

But.. if I harcode "2016" then..

year="2016"

All works fine..

Why?