Killing python ffmpeg subprocess breaks cli output

Question

I'm trying to execute a system command with subprocess and reading the output.

But if the command takes more than 10 seconds I want to kill the subprocess.

I've tried doing this in several ways.

My last try was inspired by this post: https://stackoverflow.com/a/3326559/969208

Example:

import os
import signal
from subprocess import Popen, PIPE

class Alarm(Exception):
    pass

def alarm_handler(signum, frame):
    raise Alarm

def pexec(args):

    p = Popen(args, stdout=PIPE, stderr=PIPE)

    signal.signal(signal.SIGALRM, alarm_handler)
    signal.alarm(10)

    stdout = stderr = ''
    try:
        stdout, stderr = p.communicate()
        signal.alarm(0)
    except Alarm:
        try:
            os.kill(p.pid, signal.SIGKILL)
        except:
            pass

    return (stdout, stderr)

The problem is: After the program exits no chars are shown in the cli until I hit return. And hitting return will not give me a new line.

I suppose this has something to do with the stdout and stderr pipe.

I've tried flushing and reading from the pipe (p.stdout.flush())

I've also tried with different Popen args, but might've missed something. Just thought I'd keep it simple here.

I'm running this on a Debian server.

Am I missing something here?

EDIT:

It seems this is only the case when killing an ongoing ffmpeg process. If the ffmpeg process exits normally before 10 seconds, there is no problem at all.

I've tried executing a couple of different command that take longer than 10 seconds, one who prints output, one who doesn't and a ffmpeg command to check the integrity of a file.

args = ['sleep', '12s'] # Works fine
args = ['ls', '-R', '/var'] # Works fine, prints lots for a long time
args = ['ffmpeg', '-v', '1', '-i', 'large_file.mov','-f', 'null', '-'] # Breaks cli output

I believe ffmpeg prints using \\r and prints everything on the strerr pipe. Can this be the cause? Any ideas how to fix it?

Answer 1

Well. your code surely works fine on my Ubuntu server.

(which is close cousin or brother of Debian I suppose)

I added few more lines, so that I can test your code.

import os
import signal
from subprocess import Popen, PIPE

class Alarm(Exception):
    pass

def alarm_handler(signum, frame):
    raise Alarm
def pexec(args):
    p = Popen(args, stdout=PIPE, stderr=PIPE)

    signal.signal(signal.SIGALRM, alarm_handler)
    signal.alarm(1)

    stderr = ''
    try:
        stdout, stderr = p.communicate()
        signal.alarm(0)
    except Alarm:
    print "Done!"
        try:
            os.kill(p.pid, signal.SIGKILL)
        except:
            pass

    return (stdout, stderr)

args = ('find', '/', '-name','*')
stdout = pexec(args)
print "----------------------result--------------------------"
print stdout
print "----------------------result--------------------------"

Works like a charm.

If this code works on your server, I guess problem actually lies on

command line application that you trying to retrieve data.

Answer 2

I have the same problem. I can't get a running FFmpeg to terminate gracefully from a python subprocess, so I am using <process>.kill() . However I think this means FFmpeg does not restore the mode of the tty properly (as described here: https://askubuntu.com/a/172747 )

You can get your shell back by running reset at the bash prompt, but that clears the screen so you can't see your script's output as you continue to work.

Better is to run stty echo which turns echoing back on for your shell session.

You can even run this in your script after you've nuked FFmpeg. I am doing:

ffmpeg_popen.kill()
ffmpeg_popen.wait()
subprocess.call(["stty", "echo"])

This works for me on Ubuntu with bash as my shell. YMMV, but I hope it helps. It smells hacky but it's the best solution I've found.

Answer 3

I ran into a similar issue with ffmpeg. It seems that if ffmpeg is killed using Popen.kill() it does not properly close and does not reinstate echoing on your terminal.

We can solve this using a pipe to stdin, and writing q to close ffmpeg as we would in a cli session:

p = Popen(args, stdin=PIPE stdout=PIPE, stderr=PIPE)
p.stdin.write(b"q")

It's probably preferable to use Popen.communicate in order to avoid a deadlock. The following will also work:

p = Popen(args, stdin=PIPE stdout=PIPE, stderr=PIPE)
p.communicate(b'q')

But it seems like even the following works:

p = Popen(args, stdin=PIPE stdout=PIPE, stderr=PIPE)
p.kill()

I'm not sure what causes this ffmpeg to close cleanly if it has an input pipe. Perhaps it has something to do with what causes this bug in the first place?