Pipe result from subprocess to unix sort

Question

I am calling a perl script on an external txt files from python, and printing the output to an outfile. But instead I want to pipe the output to unix's sort. Right now I am not piping, but are writing the output from the perl program first, then doing by combining my code under, and this stackoverflow answer .

import subprocess
import sys
import os

for file in os.listdir("."):

    with open(file + ".out", 'w') as outfile:
        p = subprocess.Popen(["perl", "pydyn.pl", file], stdout=outfile)
        p.wait()

Answer 1

To emulate the shell pipeline:

#!/usr/bin/env python
import pipes
import subprocess

pipeline = "perl pydyn.pl {f} | sort >{f}.out".format(f=pipes.quote(filename))
subprocess.check_call(pipeline, shell=True)

without invoking the shell in Python:

#!/usr/bin/env python
from subprocess import Popen, PIPE

perl = Popen(['perl', 'pydyn.pl', filename], stdout=PIPE)
with perl.stdout, open(filename+'.out', 'wb', 0) as outfile:
    sort = Popen(['sort'], stdin=perl.stdout, stdout=outfile)
perl.wait() # wait for perl to finish
rc = sort.wait() # wait for `sort`, get exit status

Answer 2

Just use bash. Using python just adds a level of complexity you don't need.

for file in $( ls); 
do 
    perl pydyn.pl $file | sort
done

Above is a quick and dirty example, a better alternative in terms of parsing is the following:

ls | while read file; do perl pydyn.pl "$file" | sort; done

Answer 3

Since you asked the question in python you can also pipe the result

p = subprocess.Popen("perl pydyn.pl %s | sort" % file, stdout=outfile,shell=True) 

but for this you're gonna have to make it shell=True which is not a good practice

Here's one way without making it shell=True

  p = subprocess.Popen(["perl", "pydyn.pl", file], stdout=subprocess.PIPE)
  output = subprocess.check_output(['sort'], stdin=p.stdout,stdout=outfile)
  p.wait()