os.system behaves differently w/ input from raw_input() vs file.readlines()

Question

I am running a command through os.system and can get the command needed in two different ways; from user input or from a file.

# Code works fine with this
filename = raw_input('Enter a filename:' )

# but it doesn't work if I do this:
f = open("userinput.txt").readlines()
filename = f[1] 

If I now print filename I get exactly the same output. However when filename is passed through os.system it only works in the top case. The other case prints some data I did not ask for. I would post full source code but the files are huge! Here is the a snippit.

string = "sort -n -k3,3 -k2,2 -k1,1 < "
string1 = "> orderedfile.txt"

cmd = string + filename + string1
reordering = os.system(cmd)

Answer 1

Current Behavior

readlines() returns lines with \\n on the end of them. Thus, you're splitting the code you run into two separate commands. Assuming your file is unsorted_input.txt , then, this would run:

sort -n -k3,3 -k2,2 -k1,1 < unsorted_input.txt
> orderedfile.txt

...thus, it writes the output of sort to stdout, and truncates orderedfile.txt to be empty.

The smallest possible fix is just to trim the trailing newline from the filename -- but that leaves you open to lots of other bugs: Filenames with spaces, filenames with literal quotes, filenames with command substitutions or combinations thereof will throw the original approach into chaos.

---

Preferred Approach (No Shell Required)

A correct implementation would look more like:

import subprocess

def sort_file(input_filename, output_filename):
    subprocess.call(['sort', '-n', '-k3,3', '-k2,2', '-k1,1'],
                    stdin=open(input_filename, 'r'),
                    stdout=open(output_filename, 'w'))

sort_file(
    open('userinput.txt', 'r').readlines()[1].rstrip('\n'),
    'output_file.txt',
)

Preferred Approach (Safe Shell Usage)

def sort_file(input_filename, output_filename):
    subprocess.call(
        ['sort -n -k3,3 -k2,2 -k1,1 <"$1" >"$2"', # this is the shell script to run
         '_',                                     # this becomes $0 when that script runs
         input_filename,                          # this becomes $1
         output_filename],                        # this becomes $2
        shell=True)

Note in this case that we're passing filenames out of band from code, and quoting the expansions where they're used.