通过SSH实时执行的流式python命令

Question

While ssh'ed into a bastion host, I'm executing some python commands over ssh on another server. These commands sometimes take a long time, and I can't see what the script is doing or what step it's on until it finishes executing, which can take a long time. I would like to view the logs and print statements of the script on the bastion host as the script is running.

My current code is using subprocess.Popen . The relevant parts of the code are shown below. I've heard that adding the -t flag to the ssh command is relevant for my use case?

def execute_command_over_ssh(host, command):
        process = subprocess.Popen(["ssh", host, command],
                                   stdout=subprocess.PIPE,
                                   stderr=subprocess.PIPE)
        stdout, stderr = process.communicate()
        if process.returncode != 0:
            print "There was an error executing the command ", command, " over ssh to host", host
            print stderr
            print stdout
            exit(1)
        else:
            return stdout

I'm executing a chain of different commands by calling the

execute_command_over_ssh()

method and I don't see the final output of every command until the entire script has finished executing.

Answer 1

This one will redirect stderr stream into stdout and will return you the stdout of ssh as it's produced.

import subprocess
import logging

logger = logging.basicConfig(level=logging.DEBUG)

if __name__ == '__main__':
    p = subprocess.Popen('ssh user@ip', stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    while p.poll() is None:
        for line in p.stdout:
            line = line.decode().strip('\n')
            logging.debug(line)

output:

DEBUG:root:Pseudo-terminal will not be allocated because stdin is not a terminal.
DEBUG:root:Authenticated to x.x.x.x.
DEBUG:root:Welcome to Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-51-generic x86_64)
DEBUG:root:
DEBUG:root:* Documentation:  https://help.ubuntu.com
DEBUG:root:* Management:     https://landscape.canonical.com
DEBUG:root:* Support:        https://ubuntu.com/advantage
...
...