Redirect the output of a python script to THE BEGINNING of a file

Question

I am aware of the following options to append the output to a file:

test.py:

print "Error"
print "Warning"

test.txt:

Levels:
Debug

When I do:

python test.py > test.txt

It appends at the end of the file.

However, if I want to append at the beginning of a file, so that the output of my file looks like as follows:

Levels:
Error
Warning
Debug

Is there any straightforward way of doing this possibly without manually creating a temporary file (sed -i is OK for example).

I have tried several sed approach:

sed -i '1i\`python test.py`' test.txt

But none seems to be working.

Answer 1

An easy way to do this is to use a separate, temporary file:

python test.py | cat - test.txt > tmp && mv tmp test.txt

The - means that cat uses standard input.

---

I just realised that you actually want to put the text after the first line, not right at the beginning. To do this, you could use awk:

python test.py | awk 'NR==FNR{a[++n]=$0;next}1;/Levels:/{for(i=1;i<=n;++i)print a[i]}' - test.txt

NR is the overall record (line) number and FNR is the record number of the current input. NR==FNR means that the first block only operates on the first input (which in this case is the output of your python script). The output is added to a buffer a . The next means that awk skips the rest of the commands and goes to the next line.

1 is a shorthand which causes all of the lines in the file to be printed. When the line containing "Levels:" is found, each line of the buffer a is also printed.

You can use the same trick as above to write the output to a temporary file, then overwrite the original file.

Answer 2

This should do it:

python test.py |
awk '
    NR==FNR { a[NR] = $0; next }
    { print (FNR==1 ? a[1] RS : "") $0 }
    END { for (i=2;i in a;i++) print a[i] > ARGV[1] }
' test.txt -

The above reads the first file into an array, then closes that file before opening the stdin stream coming from the pipe so from then on it can safely overwrite the first file with it's original contents from the array merged with the input from the pipe in whatever order you like.

Unlike awk '...' file > file , this is perfectly safe to do since it's awk rather than the shell deciding when to overwrite the file, it's only issue would be memory usage and speed if the original file is huge.