bash command not running through python

Question

I have the following method:

def _run_command(bash_command: str) -> str:
    """
    Run a bash command.

    Args:
        bash_command: command to be executed.
    """
    logging.info(f"Running {bash_command}.")
    process = subprocess.Popen(bash_command.split(), stdout=subprocess.PIPE)
    output, error = process.communicate()
    if error is not None:
        logging.exception(f"Error running command: {bash_command}")
        raise ValueError(error)

    return str(output)

Which I use to run shell commands through Python. It seems to work for the majority of the cases except for this one:

command = f'find {self._prefix} -name "*.txt" -type f -delete'
output = self._run_command(command)

Where self._prefix is a path such as /opt/annotations/ . I'd expect this command to delete all txt files inside that path, which is not happening. However, if I run this command find /opt/annotations/ -name "*.txt" -type f -delete directly in the terminal, everything is erased as expected. So I'm wondering if I'm missing something here.

The logging shows the expect command, however, the txts aren't erased:

2020-11-18 19:07:47 fm-101 root[10] INFO Running find /opt/annotations/ -name "*.txt" -type f -delete.

Answer 1

The problem is the quotes. You don't have files matching "*.txt" , but only files matching *.txt .

Pass your command explicitly as a list, instead of trying to generate a list from a string, to avoid this and other problems:

def _run_command(command_argv: list) -> str:
    logging.info(f"Running {command_argv}.")
    process = subprocess.Popen(command_argv, stdout=subprocess.PIPE)
    output, error = process.communicate()
    if error is not None:
        logging.exception(f"Error running command: {bash_command}")
        raise ValueError(error)
    return str(output)

_run_command(['find', str(self._prefix), '-name', '*.txt', '-type', 'f', '-delete'])

---

If you insist on taking arguments as a string, use shlex.split() to get POSIX sh-like (not bash-like) handling of quoted values:

def _run_command(shell_command: str) -> str:
    """
    Run a shell command, but without actually invoking any shell

    Args:
        shell_command: command to be executed.
    """
    logging.info(f"Running {shell_command}.")
    process = subprocess.Popen(shlex.split(shell_command), stdout=subprocess.PIPE)
    output, error = process.communicate()
    if error is not None:
        logging.exception(f"Error running command: {shell_command}")
        raise ValueError(error)

    return str(output)

...but that only fixes the specific concern of quote removal; it doesn't make operations shell-compatible in any other respect.