How to copy a complete directory recursively to a remote server in Python (paramiko) using SCP or SSH?

Question

I have a directory containing some files and sub-directories on my local machine that is generated by a daily Python script running. And then I want to copy all those generated files in a directory to the server and then run some series of commands on it by ssh using python package paramiko.

I want to add little code to securely copy that entire directory and files and sub-directories inside it to to my server over SSH/SCP using the paramiko package of python. I can send the individual files using the same but unable to send the entire directory and its content using paramiko. It gives me IOError saying it's a directory.

IOError: [Errno 21] Is a directory: 'temp'

Here is the my code:

from paramiko import SSHClient, AutoAddPolicy
from scp import SCPClient

class SSh(object):

    def __init__(self, address, username, password, port=22):
        print "Connecting to server."
        self._address = address
        self._username = username
        self._password = password
        self._port = port
        self.sshObj = None
        self.connect()
        self.scp = SCPClient(self.sshObj.get_transport())

    def sendCommand(self, command):
        if(self.sshObj):
            stdin, stdout, stderr = self.sshObj.exec_command(command)
            while not stdout.channel.exit_status_ready():
                # Print data when available
                if stdout.channel.recv_ready():
                    alldata = stdout.channel.recv(1024)
                    prevdata = b"1"
                    while prevdata:
                        prevdata = stdout.channel.recv(1024)
                        alldata += prevdata

                    print str(alldata)
        else:
            print "Connection not opened."

    def connect(self):
        try:
            ssh = SSHClient()
            ssh.set_missing_host_key_policy(AutoAddPolicy())
            ssh.connect(self._address, port=self._port, username=self._username, password=self._password, timeout=20, look_for_keys=False)
            print 'Connected to {} over SSh'.format(self._address)
            return True
        except Exception as e:
            ssh = None
            print "Unable to connect to {} over ssh: {}".format(self._address, e)
            return False
        finally:
            self.sshObj = ssh

if __name__ == "__main__":
    # Parse and check the arguments
    ssh = SSh('10.24.143.190', 'username', 'password')
    ssh.scp.put("Valigrind_BB.py") # This works perfectly fine
    ssh.scp.put("temp") # IOError over here Is a directory
    ssh.sendCommand('ls')

Thank you in advance.

Answer 1

Looking at the source code for SCP, it appears you can pass the parameter "recursive" as a bool to the put() method to specify transferring the contents of a directory recursively. Here is a link to the source code I am talking about. Try changing last part of your code to the following:

if __name__ == "__main__":
    # Parse and check the arguments
    ssh = SSh('10.24.143.190', 'username', 'password')
    ssh.scp.put("Valigrind_BB.py") # This works perfectly fine
    ssh.scp.put("temp", recursive=True) # IOError over here Is a directory
    ssh.sendCommand('ls')

Also, if you just want to transfer files, you can try rsync as an alternative. The modification to your code above should work though. I hope this helps.