Emulate pipe with several outlets. Is python fast enough for this?

Question

I have input bytes from ffmpeg. I'd like to send this output to several other ffmpeg processes. Since I can't use a Unix Pipe, Socket,.. for this (or can I?) I'm using python as the receiving end of the pipe. A subroutine then copies the received data to all previously "registered" outlets.

It feels wrong!

Specifically I have the feeling that this means a lot of memory copying. It works right now. But I'd seriously love to head about "the right way" of doing this.

    def writeData(self,data):
    """Write 'data' to all outputs"""
    if len(self.outlets) > 0:
        for outlet in self.outlets:
            outlet.writeData(data)
    else:
        self.logger.warn("Received data but no outlets registred (yet?)")

Answer 1

EDIT : Thanks to Phil, who corrected an error in an earlier version (see the comments).

Using bash , you could do

ffmpeg ... | tee >(ffmpeg ...) >(ffmpeg ...) >(ffmepg ...) > /dev/null

This will pipe the output of the first ffmpeg to the three others.

If you don't use bash , you can manually create named pipes, connect them to ffmpeg processes and use tee to pipe to the named pipes:

mkfifo pipe1 pipe2
ffmpeg ... < pipe1
ffmpeg ... < pipe2
ffmpeg ... | tee pipe1 pipe2 | ffmpeg ...
rm pipe1 pipe2

The above bash code does basically the same. The creation and removal of the named pipes happens transparently.

Answer 2

This is likely to be fast enough, especially with the speed of CPUs now compared to the relatively lower bandwidth of compressed video data.

As always, you will have to test and benchmark your solution to ensure that it meets your performance requirements.

Answer 3

Have you looked at the "pee" utility from the moreutils package? It does just what you're describing:

ffmpeg | pee 'ffmpeg -stuff1' 'ffmpeg -stuff2' 'ffmpeg -stuff3'