How to capture python subprocess stdout in unittest

Question

I am trying to write a unit test that executes a function that writes to stdout, capture that output, and check the result. The function in question is a black box: we can't change how it is writing it's output. For purposes of this example I've simplified it quite a bit, but essentially the function generates its output using subprocess.call().

No matter what I try I can't capture the output. It is always written to the screen, and the test fails because it captures nothing. I experimented with both print() and os.system(). With print() I can capture stdout, but not with os.system() either.

It's also not specific to unittesting. I've written my test example without that with the same results.

Questions similar to this have been asked a lot, and the answers all seem to boil down to use subprocess.Popen() and communicate(), but that would require changing the black box. I'm sure there's an answer I just haven't come across, but I'm stumped.

We are using Python-2.7.

Anyway my example code is this:

#!/usr/bin/env python
from __future__ import print_function
import sys
sys.dont_write_bytecode = True

import os
import unittest
import subprocess
from contextlib import contextmanager
from cStringIO import StringIO

# from somwhere import my_function
def my_function(arg):
    #print('my_function:', arg)
    subprocess.call(['/bin/echo', 'my_function: ', arg], shell=False)
    #os.system('echo my_function: ' + arg)

@contextmanager
def redirect_cm(new_stdout):
    old_stdout =  sys.stdout
    sys.stdout =  new_stdout
    try:
        yield
    finally:
        sys.stdout = old_stdout

class Test_something(unittest.TestCase):
   def test(self):
        fptr = StringIO()
        with redirect_cm(fptr):
            my_function("some_value")

        self.assertEqual("my_function: some_value\n", fptr.getvalue())

if __name__ == '__main__':
    unittest.main()

Answer 1

There are two issues in the above code

1. StringIO fptr does not shared by the current and the spawned process, we could not get the result in current process even if the spawned process has written result to StringIO object

2. Changing sys.stdout doesn't affect the standard I/O streams of processes executed by os.popen() , os.system() or the exec*() family of functions in the os module

A simple solution is

1. use os.pipe to share result between the two processes

2. use os.dup2 instead of changing sys.stdout

A demo example as following shown

import sys
import os
import subprocess
from contextlib import contextmanager


@contextmanager
def redirect_stdout(new_out):
    old_stdout = os.dup(1)
    try:
        os.dup2(new_out, sys.stdout.fileno())
        yield
    finally:
        os.dup2(old_stdout, 1)


def test():
    reader, writer = os.pipe()

    with redirect_stdout(writer):
        subprocess.call(['/bin/echo', 'something happened what'], shell=False)

    print os.read(reader, 1024)


test()