Python subprocess cannot capture output of Windows program

Question

Using stdout=subprocess.PIPE stops the output from going to the console, but nothing is captured.

>>> import subprocess
>>> proc = subprocess.Popen(['C:\\Users\\me\\program.exe'])
>>> ERROR: please provide an argument
// TRUNCATED USAGE OUTPUT
proc.wait()
0
>>> proc = subprocess.Popen([''C:\\Users\\me\\program.exe''], stdout=subprocess.PIPE)
>>> proc.communicate()
('', None)

I've tried every combination available on stackoverflow. shell=True hasn't worked. Spawning a sub cmd hasn't worked. subprocess.check_output captures nothing. I'm happy to retry any of these commands in the comments.

I am guessing this has something to do with out the program is attaching to a shell.

This is the assembly the program uses to output ( mcall is just a macro to align memory to 16 bits). The reason I include this is in case GetStdHandle is affecting things.

console_write PROC
; rcx   MSG
; rdx   LEN
  prologue
  push    rcx
  push    rdx
  xor     rcx, rcx
  mov     ecx, [stdout]
  mcall   GetStdHandle
  mov     rcx, rax
  xor     rdx, rdx
  pop     r8    ; len
  pop     rdx   ; msg
  push    0
  mov     r9, rsp
  push    0
  mcall   WriteConsoleA
  pop     rcx
  pop     rcx
  epilogue
console_write ENDP

I'm stumped on this one. I've done this so many times on Linux. I just don't know enough about Windows internals to sort this out. Thanks!

---

Edit: Additional things I have tried:

Admin (also with STDERR capture)

C:\Windows\system32>C:\Python27\python.exe
Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 20:19:30) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> proc = subprocess.Popen(['C:\\Users\\me\\program.exe'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> proc.communicate()
('', '')

STDOUT File Redirect

>>> import os
>>> os.system('C:\\Users\\me\\program.exe > out.txt')
0
>>> f = open('out.txt', 'r')
>>> f.read()
''

STDERR File Redirect

>>> import os
>>> os.system('C:\\Users\\me\\program.exe 2>out.txt')
ERROR: please provide an argument
// TRUNCATED USAGE OUTPUT
0
>>> open('out.txt', 'r').read()
''

Command Line Capture Fails (suppresses output from going to cmd, but nothing is captured)

program.exe > out.txt

Answer 1

A coworker pointed out the issue. It turns out that WriteConsole cannot be redirected to a file, it can only be used with console handles. [1] [2]

This StackOverflow answer gives an outline for how to solve it. I've implemented it in Python if anyone else has this problem.

import win32console
import subprocess
import msvcrt
import os

ccsb = win32console.CreateConsoleScreenBuffer()
fd = msvcrt.open_osfhandle(ccsb, os.O_APPEND)
proc = subprocess.Popen(['C:\\Users\\me\\program.exe'], stdout=fd)
proc.wait()
ccsb.ReadConsoleOutputCharacter(100, win32console.PyCOORDType(0,0)) # reads 100 characters from the first line

The better solution for me was to change WriteConsoleA to WriteFile , using the same StdHandle .

console_write PROC
; rcx   MSG
; rdx   LEN
  prologue
  push    rcx
  push    rdx
  xor     rcx, rcx
  mov     ecx, [stdout]
  mcall   GetStdHandle
  mov     rcx, rax
  ; Write File
  pop     r8    ; len
  pop     rdx   ; msg
  ; unalign for odd number of pushed args
  mov     rbx, rsp
  sub     rbx, 8        
  and     rbx, 0Fh
  sub     rsp, rbx
  ; args
  mov     rcx, rax        ; handle
  xor     r9, r9
  push    0               ; 1 arg (ununaligns)
  sub     rsp, 20h        ; shadow 
  call    WriteFile
  add     rsp, 28h        ; shadow + arg
  add     rsp, rbx        ; realign
  epilogue
console_write ENDP

Answer 2

Did you try to catch stderr as well?

proc = subprocess.Popen([''C:\\Users\\me\\program.exe''], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

UPDATE

You can try to read output line by line while the process is running, for example:

p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while True:
    retcode = p.poll()
    print p.stdout.readline()
    if retcode is not None:
        break

Answer 3

I don't think I have enough rep to suggest a question is a possible duplicate, nor comment, so I'm posting this as an answer because although douggard's answer led me down the right track thinking-wise, it didn't actually work in my case and I had a hard time getting a working solution.

There's an answer here that captures CONOUT and worked for me: https://stackoverflow.com/a/38749458/1675668