How to store output from linux call in Python

Question

I am writing a script to sort a file based on a specific column. I tried calling the 'sort' Linux command for this. The code I am using is:

from subprocess import 
path_store = /homes/varshith/maf
input = path_store
field = "-k2"
store_output_in_new_file = ">"
new_path = path_store + "_sort.bed"
sorting = Popen(["sort", field, input, append, new_path], stdout=PIPE)

But this doesn't work properly. Thanks in advance for helping.

Answer 1

Use communicate to get the output:

from subprocess import PIPE,Popen
sorting = Popen(["sort", field, output, append, new_path], stdout=PIPE)
out, err = sorting.communicate()  
print out

Or just use check_output for python >= 2.7:

sorting = check_output(["sort", field, output, append, new_path])

If you want to just write the sorted contents you can redirect stdout to a file object:

output = "path/to/parentfile"
cmd = "sort -k2 {}".format(output)
with open(new_file,"w") as f:
    sorting = Popen(cmd.split(),stdout=f)

Answer 2

First of all, I hope that output and new_path are actually strings (I assume so, but it's not clear from what you posted). But assuming all of that is sorted out:

sorting = Popen(...)
sorting_output = sorting.communicate()[0]

That should store the contents of the subprocess's standard output to sorting_output .

Answer 3

To emulate the shell command:

$ sort -k2  /homes/varshith/maf > /homes/varshith/maf_sort.bed

ie, to sort /homes/varshith/maf file by the 2nd column and to store the sorted output to /homes/varshith/maf_sort.bed file in Python:

#!/usr/bin/env python
from subprocess import check_call

field = '-k2'
input_path  = '/homes/varshith/maf'
output_path = input_path + '_sort.bed'
with open(output_path, 'wb', 0) as output_file:
    check_call(["sort", field, input_path], stdout=output_file)

It overwrites the output file. To append to the file instead, use ab mode instead of wb .