plumbum.commands.processes.ProcessExecutionError: for commands which return null

Question

The shell command I want to run, which returns nothing:

echo helloWorld | grep 'dummy'

---

plumbum version:

Following line works:

out=(echo["helloWorld"] | grep["h"])().strip()

---

But following line does not, what might be the reason?

out=(echo["helloWorld"] | grep["dummy"])().strip()
print(out)

Error I am having:

Traceback (most recent call last):
  File "dd.py", line 6, in <module>
    out=(echo["helloWorld"] | grep["dummy"])().strip()
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/base.py", line 103, in __call__
    return self.run(args, **kwargs)[1]
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/base.py", line 240, in run
    return p.run()
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/base.py", line 201, in runner
    return run_proc(p, retcode, timeout)
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/processes.py", line 232, in run_proc
    return _check_process(proc, retcode, timeout, stdout, stderr)
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/processes.py", line 23, in _check_process
    proc.verify(retcode, timeout, stdout, stderr)
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/base.py", line 412, in verify
    dstproc_verify(retcode, timeout, stdout, stderr)
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/machines/base.py", line 26, in verify
    stderr)
plumbum.commands.processes.ProcessExecutionError: Command line: ['/bin/grep', 'dummy']
Exit code: 1

[Q] How could I fix this error?

Answer 1

This happens because the exit status of grep is 1 if it does not find anything, as described in its manual

You can try it in command line if you wish:

echo helloWorld | grep h; echo $?
echo helloWorld | grep x; echo $?

Will result in

helloWorld
0
1

Ways to circumvent this are described in another nice answer , eg

echo helloWorld | grep x | cat

will yield 0 as status. But unfortunately plumbum does a local pipe mechanism so grep output goes to plumbum, and then plumbum pipes it to the next command - meaning the cat can't swallow the exit code 1, there will be an exception thrown before it.

So my two ideas are to either create a shell script to run a grep never returning error on search without results:

#!/bin/bash
grep "$@" || test $? = 1

and execute this instead of the grep (called c1grep in the original answer), or to add a try/except block around your pipe code and manually handle the exit code 1 (ProcessExecutionError).

Answer 2

The other answer is correct but less than optimal - The problem can be solved in plumbum, without the need for an external script that wraps grep , and without having to catch the exception.

The idea is to use plumbum's run method to to do two things:

• Force plumbum to accept any exit code from the called program (by default, an exception is raised if the returned error code is not zero).
• Capture a tuple containing the error code, the standard output, and the standard error of the called program, and not just the standard output as plumbum normally does.

Working example:

for s in ["hello","dummy"]:
    exitcode,stdout,stderr = (echo["helloWorld"] | grep[s]).run (retcode=None)

    if exitcode == 0:
        print (stdout.strip())

    if exitcode != 0:
        print (f"string {s} not present")
        print (stderr)

The previous code returns:

helloWorld
string dummy not present

Without raising any exception.