如何使用许多参数执行python subprocess.Popen？

Question

I need to execute the same command on a local and remote server. So I'm using subprocess.Popen to execute, and local command work as expected, but when I execute on remote it gives me some error like command not found. I appreciate your support as I am new to this.

Local execution function

    def topic_Offset_lz(self):
         CMD = "/dsapps/admin/edp/scripts/edp-admin.sh kafka-topic offset %s -e %s | grep -v Getting |grep -v Verifying | egrep -v '^[[:space:]]*$|^#' | awk -F\: '{print $3}'|sed '%sq;d'" % (self.topic,self.envr,self.partition)
         t_out_lz, t_error_lz = subprocess.Popen(CMD, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
         return t_out_lz

Remote server execution

    def topic_offset_sl(self):
         CMD = "/dsapps/admin/edp/scripts/edp-admin.sh kafka-topic offset %s -e %s | grep -v Getting |grep -v Verifying | egrep -v '^[[:space:]]*$|^#' | awk -F\: '{print $3}'|sed '%sq;d'" % (self.topic, self.envr, self.partition)
         t_out_sl, t_error_sl = subprocess.Popen(["ssh", "-q", CMD], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
         return t_error_sl

Error I'm getting for the remote execution

Landing Zone Offset: 0

SoftLayer Zone Offset: /bin/sh: ^# |sed 1: command not found /bin/sh: d: command not found

Answer 1

The ssh command passes its argument vector as a single command line string, not an array. To do this, it simply concatenates the arguments, without performing shell quoting:

$ ssh target "python -c 'import sys;print(sys.argv)'" 1 2 3
['-c', '1', '2', '3']
$ ssh target "python -c 'import sys;print(sys.argv)'" "1 2 3"
['-c', '1', '2', '3']

If there was proper shell quoting, the distinction between 1 2 3 and "1 2 3" would have been preserved, and the first argument would not need double-quoting.

Anyway, in your case, the following might work:

def topic_offset_sl(self):
     CMD = "ssh -q " + pipes.quote("/dsapps/admin/edp/scripts/edp-admin.sh"
             + " kafka-topic offset %s -e %s" % (self.topic, self.envr)) \
         + "grep -v Getting |grep -v Verifying | egrep -v '^[[:space:]]*$|^#'"
         + " | awk -F\: '{print $3}'|sed '%sq;d'" % self.partition
     t_out_sl, t_error_sl = subprocess.Popen(CMD], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
     return t_error_sl

This assumes you only want to run the /dsapps/admin/edp/scripts/edp-admin.sh script remotely and not the rest.

Note that the way you use string splicing to construct command lines likely introduces shell command injection vulnerabilities (both locally and on the remote server).

Answer 2

I came up with below solution, might be there will easy way rather than this.

 def topic_offset_sl(self):
        CMD_SL1 = "ssh -q %s '/dsapps/admin/edp/scripts/edp-admin.sh kafka-topic offset %s -e %s'" % (KEY_SERVER,self.topic, self.envr)
        CMD_SL2 = "| grep -v Getting |grep -v Verifying | egrep -v '^[[:space:]]*$|^#' | awk -F\: '{print $3}'|sed '%sq;d'" % (self.partition)
        t_out_sl, t_error_sl = subprocess.Popen(CMD_SL1 + CMD_SL2 , stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True).communicate()
        return t_out_sl