如何从 python 脚本中捕获多个文件的内容

Question

I would like to cat the contents of the files generated from a python script. Is it possible to do that in a simple one line command? For example I would like to have something like:

cat <(python test.py) # doesnt work as I want to

where test.py produces multiple filenames like so (separated by new line)

file1.txt
file2.txt
file3.txt

I would like to basically do

cat file1.txt
cat file2.txt
cat file3.txt

Basically reading the contents of the filename produced by the script. Assume the python script can generate hundreds/thousands of filenames.

Though this may seem to work

cat $(python test.py)

But the problem is it seems to wait until the whole python test.py is completed, before it performs any cat . Basically it doesnt seem to cat the contents of the filename as soon as it gets a filename. Where as

cat <(python test.py)

cat the filename as it gets it, unfortunately, it just prints the filename but not the content of the filename.

Answer 1

You could use sed

$ sed 's/^/cat /e' <(python3 test.py)

This will add cat in front of each filename before executing the command.

^ - This will anchor the find to the start of each line

cat - cat will replace the anchor at the start of each line

e - This tells sed to execute the commmand that resulted from the substitution, in this case cat file1.txt

Answer 2

I think you need to create the STDOUT with these files in your script:

For example test.py

import os
for i in range(0,5):
  name="file" + str(i) +".txt"
  f = open(name, "a")
  f.write("Hello\n")
  f.close()
  print(name) 

Something like this:

$ python test.py
file0.txt
file1.txt
file2.txt
file3.txt
file4.txt

$ cat $(python test.py)
Hello
Hello
Hello
Hello
Hello

If you want the cat to work on the fly, it's not so simple because in the single line bash we have to wait for the previous command to finish. But you can try to do something like this:

$ python test.py > /dev/null &
$ watch -n60 'find ./  -maxdepth 1 -type f -mmin -1 -exec cat {} \;'

Answer 3

Other than using sed , as in this answer , you should consider xargs :

python3 test.py | xargs -i cat "{}"

This is more robust than cat since, unlike the sed solution, it works well with many unconventional characters such as '*' , '?' , and ' ' (but not with '\n' ).

A minor change to your python script, it is trivial to make it work well with filenames with '\n' characters as well. The change in the python scripts will use '\0' instead of '\n' to separate the filenames (make sure to flush stdout after each such filename). Then use xargs with the -0 argument:

python3 test.py | xargs -0 -i cat "{}"