python sub-process

Question

I usually execute a Fortran file in Linux (manually) as:

1. Connect to the server
2. Go to the specific folder
3. ifort xxx.for -o xxx && ./xxx (where 'xxx.for' is my Fortran file and 'xxx' is Fortran executable file)

But I need to call my fortran file (xxx.for) from python (I'm a beginner), so I used subprocess with the following command:

cmd = ["ssh", sshConnect, "cd %s;"%(workDir), Fortrancmd %s jobname "%s -o  %s" exeFilename "%s && %s ./ %s%s"%(exeFilename)]

But I get an error, and I'm not sure what's wrong. Here's the full code:

import string
import subprocess as subProc
from subprocess import Popen as ProcOpen
from subprocess import PIPE
import numpy
import subprocess

userID = "pear"
serverName = "say4"
workDir = "/home/pear/2/W/fortran/"
Fortrancmd = "ifort"
jobname = "rad.for"
exeFilename = "rad"

sshConnect=userID+"@"+servername

cmd=["ssh", sshConnect, "cd %s;"%(workDir), Fortrancmd %s jobname "%s -o  %s" exeFilename "%s && %s ./ %s%s"%(exeFilename)]

**#command to execute fortran files in Linux
**#ifort <filename>.for -o <filename> && ./<filename> (press enter)

**#example:ifort xxx.for -o xxx && ./xxx (press enter)

print cmd

How can I write a python program that performs all 3 steps described above and avoids the error I'm getting?

Answer 1

there are some syntax errors...

original:

cmd=["ssh", sshConnect, "cd %s;"%(workDir), Fortrancmd %s jobname "%s -o  %s" exeFilename "%s && %s ./ %s%s"%(exeFilename)]

I think you mean:

cmd = [
      "ssh",
      sshConnect,
      "cd %s;" % (workDir,),
      "%s %s -o  %s && ./%s" % (Fortrancmd, jobname, exeFilename, exeFilename)
      ]

A few notes:

• a tuple with one element requires a comma at the end of the first argument see (workDir,) to be interpreted as a tuple (vs. simple order-of-operations parens)
• it is probably easier to contruct your fortan command with a single string format operation

PS - For readability it is often a good idea to break long lists into multiple lines :)

my advice

I would recommend looking at this stackoverflow thread for ssh instead of using subprocess

Answer 2

For the manual part you may want to look into pexpect or for windows wexpect . These allow you to perform subprocesses and pass input under interactive conditions.

However most of what you're doing sounds like it would work well in a shell script. For simplicity, you could make a shell script on the server side for your server side operations, and then plug in the path in the ssh statement:

ssh user@host "/path/to/script.sh"

Answer 3

You could use fabric for steps 1 and 2. This is the basic idea:

from fabric.api import *
env.hosts = ['host']
dir = '/home/...'

def compile(file):
     with cd(dir):
         run("ifort %s.for -o %s" %(file,file))
         run("./%s > stdout.txt" % file)

• Create fabfile.py
• And you run fab compile:filename

Answer 4

one error:
you have an unquoted %s in your list of args, so your string formatting will fail.

Answer 5

Here is a complete example of using the subprocess module to run a remote command via ssh (a simple echo in this case) and grab the results, hope it helps:

>>> import subprocess
>>> proc = subprocess.Popen(("ssh", "remoteuser@host", "echo", "1"), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> stdout, stderr = proc.communicate()

Which in this case returns: ('1\\n', '')

Note that to get this to work without requiring a password you will likely have to add your local user's public key to ~remoteuser/.ssh/authorized_keys on the remote machine.

Answer 6

do you have to use python?

ssh user@host "command"