How to use Popen using Python on Windows?

Question

I am trying out an example Pipe program using Python on Windows

import subprocess

p1 = subprocess.Popen(["powershell", "Get-ChildItem C:\\dev\\python"],  stdout=subprocess.PIPE);

p2 = subprocess.Popen(["powershell", "Select-String", "py"], stdin=p1.stdout, stdout=subprocess.PIPE);
p1.stdout.close();

p2_output = p2.communicate()[0];
print(p2_output);

However, it errors with the following

cmdlet Select-String at command pipeline position 1
Supply values for the following parameters:
Path[0]: b"\r\nSelect-String : Cannot bind argument to parameter 'Path' because it is an empty array.\nAt line:1 char:1\n+ Select-String py\n+ ~~~~~~~~~~~~~~~~\n    + CategoryInfo          : InvalidData: (:) [Select-String], ParameterBindingValidationException\n    + FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyArrayNotAllowed,Microsoft.PowerShell.Commands.SelectStringCommand\n \n"

I expect the program to work as 'stdin' for P2 takes the output of P1's 'stdout'. Not sure what am I doing wrong ?

Answer 1

You use Popen correctly, however PowerShell's pipeline between 2 command-lets is very different in its nature from a regular pipeline between processes. PowerShell transfers objects. Look more in the Understanding pipeline document.

Unfortunately this feature cannot be used for communication between separate PowerShell processes using system pipeline.

So this command will fail exactly the same way as it fails in your code:

powershell Get-ChildItem C:\\dev\\python | powershell Select-String py

cmdlet Select-String at command pipeline position 1
Supply values for the following parameters:
Path[0]:
Select-String : Cannot bind argument to parameter 'Path' because it is an empty array.
At line:1 char:1
+ Select-String -Pattern py
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Select-String], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyArrayNotAllowed,Microsoft.PowerShell.Commands.SelectStringCommand

You should use Popen to work with the single PowerShell process and let PowerShell to handle pipelining:

import subprocess

p = subprocess.Popen("powershell Get-ChildItem C:\\dev\\python | Select-String py", stdout=subprocess.PIPE)

p_output = p.communicate()[0].decode()
print(p_output)

PS Don't use ; in Python