How to add timestamp while redirecting stdout to file in Bash?

Question

I have a program (server) and I am looking for a way (script) that will redirect (or better duplicate) all its stdout to file and add timestamp for each entry.

I've done some research and the furthest I could get was thanks to How to add timestamp to STDERR redirection . It redirects stdout but the timestamp added is of the time when the script finishes:

#!/bin/bash
./server | ./predate.sh > log.txt

code of predate.sh :

#!/bin/bash
while read line ; do
    echo "$(date): ${line}"
done

It seems that server output is flushed after exit of the program.(without redirecting it works fine). Also if I try using predate.sh on given example in mentioned thread, it works perfectly. I am aware it would be easy adding a timestamp to the main program but I would rather avoid editing its code.

Answer 1

I recently needed exactly that: receive log messages in a serial console (picocom), print them to a terminal and to a file AND prepend the date.

What I now use looks s.th. like this:

picocom -b 115200 /dev/tty.usbserial-1a122C | awk '{ print strftime("%s: "), $0; fflush(); }' | tee serial.txt

• the output of picocom is piped to awk
• awk prepends the date (the %s option converts the time to the Number of seconds since 1970-01-01 00:00:00 UTC - or use %c for a human-readable format)
• fflush() flushes any buffered output in awk
• that is piped to tee which diverts it to a file. (you can find some stuff about tee here )

Answer 2

moreutils ts

Absolute date and time is the default:

$ sudo apt-get install moreutils
$ (echo a;sleep 1;echo b;sleep 3;echo c;sleep 2;echo d;sleep 1) | ts | tee myfile
$ cat myfile
Apr 13 03:10:44 a
Apr 13 03:10:45 b
Apr 13 03:10:48 c
Apr 13 03:10:50 d

or counting from program start with ts -s :

$ (echo a; sleep 1; echo b; sleep 3; echo c; sleep 2; echo d; sleep 1) | ts -s
00:00:00 a
00:00:01 b
00:00:04 c
00:00:06 d    

or deltas for benchmarking with ts -i :

$ (echo a; sleep 1; echo b; sleep 3; echo c; sleep 2; echo d; sleep 1) | ts -i
00:00:00 a
00:00:01 b
00:00:03 c
00:00:02 d
$ (echo a; sleep 1; echo b; sleep 3; echo c; sleep 2; echo d; sleep 1) | ts -i '%.s'
0.000010 a
0.983308 b
3.001129 c
2.001120 d

See also: How to monitor for how much time each line of stdout was the last output line in Bash for benchmarking?

Tested on Ubuntu 18.04, moreutils 0.60.

Answer 3

For Me Your Code is working perfectly fine

Check this is what I tried

test.sh

#!/bin/bash

while true; do
  echo "hello"
done

predate.sh

#!/bin/bash

while read line; do
  echo $(date) ":" $line;    
done

then

./test.sh  | ./predate.sh

gives me

Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello
Tue Jan 14 17:49:47 IST 2014 : hello

This can be redirected to some file using ">" or ">>" for append

Answer 4

Again, using ts from moreutils , you can just use exec at the top of your script.

#!/bin/bash

exec > >(ts>>file.log)

echo hello 1
echo hello 2
sleep 5
echo hello 3

Answer 5

If I understand your problem is to have stderr output included in your log.txt file. Right ? If that's what you want the solution is:

./server 2>&1 | ./predate.sh > log.txt

Regards