How to debug code run using Popen in Pycharm

Question

I am running a codebase I have inherited from someone else which makes extensive use of user input. For this reason I am running it using subprocess.Popen . Here is an example. The following script ( caller.py ) calls the third-party code.

from subprocess import Popen, PIPE, STDOUT
import sys
user_input = ['John', '555']

communicate_argument = '\n'.join(user_input)
p = Popen([sys.executable, 'example2.py'], stdout=PIPE, stdin=PIPE, stderr=STDOUT, encoding='utf-8')

stdout, stderr = p.communicate(communicate_argument)

print(stdout)

The following script ( example.py ) emulates behavior of the source code I was provided, by accepting a couple of input arguments from the user:

name = input('What is your name\n')
age = input('What is your age\n')

print('You are {}, and you are {} years old'.format(name, age))

Running the code works fine, and I get the expected output.

Debugging the code partially works, but partially doesn't. The debugger successfully attaches to the child process p such that any breakpoints placed in example.py will work. However it seems that the debug console does not successfully attached to the child process. When I try to enter some variables in the debug console, they do not print out, even if they appear as active variables in my debug session.

EDIT

It turns out this might be a bug. I have asked the same question in pycharm's official forum and they made an issue out of it:

https://intellij-support.jetbrains.com/hc/en-us/community/posts/360009571680-Debugging-code-run-through-subprocess-Popen

So I guess what I'd be looking for is an effective workaround which would allow me to use a regular python interpreter in combination with the debugger to inspect and run operations on variables.

Answer 1

I have had a similar issue with python and docker logs. What solved was running python with the -u flag as described here . Perhaps try changing your Popen call to use python and then include the -u flag. eg Popen(["python", "-u", "example2.py"], stdout=PIPE, stdin=PIPE, stderr=STDOUT, encoding='utf-8')