Creating an installation bash script with the python csv module

Question

The following code should give me a "working bash script"

box_category=GET_COMBOBOX_value(self,self.combobox_category2,False)
list_path=Data_Location+"/program-data/programs_to_install.csv"
list=csv.reader(open(list_path),delimiter='|')

bash_file=path+"/install_programs"
if os.path.exists(bash_file):
    os.remove(bash_file)

with open(bash_file, "ab") as f:
    bash_list=csv.writer(f,delimiter='|')   
    bash_list.writerow(["#!/bin/bash"])
    bash_list.writerow([" "])
    string="if [ \"$(id -u)\" != \"0\" ]; then echo \"You need root permission for installing programs.\" 1>&2; exit 1; fi"
    print string        
    bash_list.writerow([string])

    for row in list :
        bash_list.writerow(["apt-get --yes install "+row[0]])

time.sleep(0.2)
os.system("chmod a+x '"+bash_file+"'")

At the first look, it is "almost" correctly, it generates me this bash file:

#!/bin/bash

"if [ ""$(id -u)"" != ""0"" ]; then echo ""You need root permission for installing programs."" 1>&2; exit 1; fi"
apt-get --yes install 0ad
apt-get --yes install audacity
apt-get --yes install banshee
apt-get --yes install barrage
..etc..

but:

• When i execute the bash file, i get the error bash: ./install_programs: /bin/bash^M: bad interpreter: No such file or directory

If i erase everything until the first apt-get .. line, and add the bash extension ".sh" the interpreter recognize it.

• the csv module is not writing good

"if [ \\"$(id -u)\\" != \\"0\\" ]; then echo \\"You need root permission for installing programs.\\" 1>&2; exit 1; fi"

it should write:

if [ "$(id -u)" != "0" ]; then echo "You need root permission for installing programs." 1>&2; exit 1; fi

• And my last mystery, is that even if i fix all those things manually, once i execute the script it doesn't installs any program, even if the commands are right, and work if i use them in a termial..

i get an outputs like this:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package 0ad
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package audacity
Reading package lists... Done
Building dependency tree       
Reading state information... Done

Answer 1

The whole point of CSV is that it's for Comma-Separated Values—that is, multiple columns on each line, separated by commas. Which means that if any of your values happens to have a comma in it, that value has to be quoted (or escaped). And those quoting rules mean that if any of your values happens to have a quote character in it, the value has to be quoted to deal with that. So, instead of this:

if [ "$(id -u)" != "0" ]; then echo "You need root permission for installing programs." 1>&2; exit 1; fi

… you end up with this:

"if [ ""$(id -u)"" != ""0"" ]; then echo ""You need root permission for installing programs."" 1>&2; exit 1; fi"

Which is of course not a valid bash command.

The easy solution is to just not use the csv module; just write your lines to the file, instead of trying to turn each one into a list of one column for the csv module to write in hopes of getting the line you wanted.

with open(bash_file, "a") as f:
    f.write("#!/bin/bash\n")
    f.write(" \n")
    string="if [ \"$(id -u)\" != \"0\" ]; then echo \"You need root permission for installing programs.\" 1>&2; exit 1; fi"
    print string        
    f.write(string + "\n")
    for row in list:
        f.write("apt-get --yes install " + row[0] + "\n")

---

If you're getting sick of typing all those explicit newlines all over, you can wrap the write function up:

def writeline(f, line):
    f.write(line + '\n')

And then, everywhere you did this:

f.write(string + "\n")

… you can instead do this:

writeline(f, string)

This is probably a good idea; otherwise, you're going to forget the "\\n" on one line—or, worse, mistype it—and spend an hour debugging your code that's obviously correct no matter how much you stare at it, and yet still doesn't work…

---

This error:

bash: ./install_programs: /bin/bash^M: bad interpreter: No such file or directory

… is also caused by using the csv module.

The default CSV dialect uses Windows-style newlines—that is, each row ends in '\\r\\n' . (See Dialect.lineterminator in the documentation.) But bash wants Unix-style newlines—that is, each line ends in '\\n' . So, it treats that extra '\\r' as if it were part of the actual line. So, on the shebang line, instead of thinking you want '/bin/bash' as your interpreter, it thinks you want '/bin/bash\\r' —which is a perfectly valid filename, but there's no file with that name.

---

And your last problem could very well be exactly the same problem. An extra invisible '\\r' character may be preventing the fi\\r from matching the if , or causing you to try to install a package named '0ad\\r' instead of '0ad' , etc.

---

A couple of side notes:

• Never call a variable list ; that hides the built-in type/function of the same name, which you may want to use. Also, although it comes up less often, string is the name of a stdlib module, so usually avoid that too.
• If you write your string literal with single quotes instead of double, you don't need to escape every double-quote inside of it. Or you can just use tripled (single or double) quotes, and you don't need to escape either kind of quotes, or even newlines: """if [ "$(id -u)" != "0" ]; then echo "You need root permission for installing programs." 1>&2; exit 1; fi"""

Answer 2

The first clue is probably in the "^M" . This is a carriage return, and is regarded by the shebang execution mechanism to be part of the executable path. Don't use text files created in Windows in other environments without appropriate sanitization. You overcame that by using sh to run the script anyway. As @abarnert points out in his/her answer, the csv module isn't required for output and indeed gets in the way.