python3.6 - TypeError: write() argument must be str, not bytes - but no files involved

Question

the following code returns an error and I don't understand why... Running on Python 3.6

import subprocess
import sys
import os

def execute_shell_cmd(cmd):
        process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
        for c in iter(lambda: process.stdout.read(1), b''):
                sys.stdout.write(c)
        for e in iter(lambda: process.stderr.read(1), b''):
                sys.stderr.write(e)

execute_shell_cmd("ls -l")

The error returned: TypeError: write() argument must be str, not bytes

Everything I see online talks about files and opening them with 'wb' options, but this is irrelevant here.

I'm sure it's silly... Any ideas?

Answer 1

You're opening the subprocess without an encoding parameter set, so the streams are binary streams (which is an excellently sane default, considering eg something like GhostScript could output a binary PDF on stdout ).

Do

process = subprocess.Popen(
   cmd,
   stdout=subprocess.PIPE, 
   stderr=subprocess.PIPE,
   shell=True,
   encoding='utf-8',
   errors='strict',   # could be ignore or replace too, `strict` is the default
)

if you'd like the streams to be wrapped in an UTF-8 decoder so you get strings out of them, not bytes. Of course, this implies you know the output data is always UTF-8.