python shell command pipes grep no results

Question

I am trying to get a domain name from a cpanel user in python.

Here is my code:

import subprocess
user = "testuser"
getuserdata = 'cat /var/cpanel/users/' + user
getdnsline = 'grep "DNS="'
test = 'wc -l'

userdataprocess = subprocess.Popen(getuserdata.split(), stdout=subprocess.PIPE)

testprocess = subprocess.Popen(test.split(), stdin=userdataprocess.stdout, stdout=subprocess.PIPE)

test, error = testprocess.communicate()
print(test)

dnslineprocess = subprocess.Popen(getdnsline.split(), stdin=userdataprocess.stdout, stdout=subprocess.PIPE)

website, error = dnslineprocess.communicate()
print(website.decode('utf-8').splitlines())

my output is:

b'60\n'
[]

So this means, that the wc -l command gives back 60 lines. So passing the output of the first getuserdata command to the wc -l command works.

However, the grep command always return blank. No matter it I put in "DNS=" or "=" or even "a". The file is the normal cpanel user file, and I have verified that DNS is in the file.

When I just output the data from the first process userdataprocess I can manually check for the DNS entry.

Do I have to do anything different when using the grep command in this fashion?

Answer 1

This is because your testprocess.communicate() for the wc -l command already consumes all of the output of userdataprocess.stdout and closes it in fact, so there's nothing left for dnslineprocess.communicate() to read.

You should instead read the output of userdataprocess.stdout into a variable and then use it as an input to both testprocess.communicate() and dnslineprocess.communicate() .

Also, as @pyb pointed out, you are unnecessarily quoting DNS= in your grep command, which, without a shell, will be passed to grep with double quotes included as part of the string to filter with. You should simply remove them as there are no special characters in your filter string.

import subprocess
user = "testuser"
getuserdata = 'cat /var/cpanel/users/' + user
getdnsline = 'grep DNS='
test = 'wc -l'

userdataprocess = subprocess.Popen(getuserdata.split(), stdout=subprocess.PIPE)

userdata = userdataprocess.stdout.read()

testprocess = subprocess.Popen(test.split(), stdin=subprocess.PIPE, stdout=subprocess.PIPE)

test, error = testprocess.communicate(userdata)
print(test)

dnslineprocess = subprocess.Popen(getdnsline.split(), stdin=subprocess.PIPE, stdout=subprocess.PIPE)

website, error = dnslineprocess.communicate(userdata)
print(website.decode('utf-8').splitlines())

Answer 2

Your script fails because of the quotes around DNS= .

You can use shell=True to make the script work:

dnslineprocess = subprocess.Popen(getdnsline, stdin=userdataprocess.stdout, stdout=subprocess.PIPE, shell=True)

Source: Passing double quote shell commands in python to subprocess.Popen()?