python subprocess remote find -exec filename with space

Question

So I have a remote Linux Server and would like to run a Python Script on my local machine to list all files and their modification dates in a specific folder on that remote server. That is my code so far:

command = "find \""+to_directory+'''\"* -type f -exec sh -c \"stat -c \'%y:%n\' \'{}\'\" \;'''

scp_process_ = subprocess.run("ssh "+to_user+"@"+to_host+" '"+command+"' ", shell=True, capture_output=False, text=True)

Now running the command

find "/shares/Public/Datensicherung/"* -type f -exec sh -c "stat -c '%y:%n' '{}'" \;

on the server itself works fine without any error.

But as soon I use a subprocess to run it remotely over ssh it has a problem with a file in a folder with spaces: "/shares/Public/Datensicherung/New folder/hi.txt" with a space in it:

stat: can't stat '/shares/Public/Datensicherung/New': No such file or directory
stat: can't stat 'folder/hi.txt': No such file or directory 

I know it is messed up, but that is the best solution I could build. I would like to stick with subprocess and ssh but if you have a better solution feel free to post it.

Answer 1

With shell=True you are invoking three shell instances, each of which requires a layer of quoting. This is possible to do, of course, but there are many reasons to avoid it if at all possible.

First off, you can easily avoid the local shell=True and this actually improves the robustness and clarity of your Python code.

command = "find \""+to_directory+'''\"* -type f -exec sh -c \"stat -c \'%y:%n\' \'{}\'\" \;'''

scp_process_ = subprocess.run(
    ["ssh", to_user+"@"+to_host, command],
    capture_output=False, text=True)

Secondly, stat can easily accept multiple arguments, so you can take out the sh -c '...' too.

command = 'find "' + to_directory + '" -type f -exec stat -c "%y:%n" {} +'

The optimization also switches + for \; (so the sh -c '' wrapper was doubly useless anyway).

Answer 2

Sometimes the issue happening because malformed command string. For purpose of comunication with Unix shell was craeted shlex module. So basically you wrap your code with shlex and then pass it into supbrocess.run .

I don't see the actual final cmd to call but you could split it to proper command with shlex.split by yourself.

From your example it would be something like:

from shlex import join

cmd = join(['ssh',
 f'{to_user}@{to_host}',
 'find',
 f'{to_directory}*',
 '-type',
 'f',
 '-exec',
 'sh',
 '-c',
 "stat -c '%y:%n' '{}'",
 ';']))

scp_process_ = subprocess.run(cmd, shell=True, capture_output=False, text=True)

Also, you maybe want to play around with shell=True option.