使用 gstreamer 1.0 使用“tee”命令将音频 + 视频 stream 录制到文件中

Question

我正在尝试将特定的音频+视频流录制到文件中（而此 stream 显示在同一台机器上）。 这是我的 receiver.sh 文件中的代码：

#!/bin/sh
#
# A simple RTP receiver
#
#  receives H264 encoded RTP video on port 5000, RTCP is received on  port 5001.
#  the receiver RTCP reports are sent to port 5005
#  receives alaw encoded RTP audio on port 5002, RTCP is received on  port 5003.
#  the receiver RTCP reports are sent to port 5007
#
#             .-------.      .----------.     .---------.   .-------.   .-----------.
#  RTP        |udpsrc |      | rtpbin   |     |h264depay|   |h264dec|   |xvimagesink|
#  port=5000  |      src->recv_rtp recv_rtp->sink     src->sink   src->sink         |
#             '-------'      |          |     '---------'   '-------'   '-----------'
#                            |          |
#                            |          |     .-------.
#                            |          |     |udpsink|  RTCP
#                            |    send_rtcp->sink     | port=5005
#             .-------.      |          |     '-------' sync=false
#  RTCP       |udpsrc |      |          |               async=false
#  port=5001  |     src->recv_rtcp      |
#             '-------'      |          |
#                            |          |
#             .-------.      |          |     .---------.   .-------.   .-------------.
#  RTP        |udpsrc |      | rtpbin   |     |pcmadepay|   |alawdec|   |autoaudiosink|
#  port=5002  |      src->recv_rtp recv_rtp->sink     src->sink   src->sink           |
#             '-------'      |          |     '---------'   '-------'   '-------------'
#                            |          |
#                            |          |     .-------.
#                            |          |     |udpsink|  RTCP
#                            |    send_rtcp->sink     | port=5007
#             .-------.      |          |     '-------' sync=false
#  RTCP       |udpsrc |      |          |               async=false
#  port=5003  |     src->recv_rtcp      |
#             '-------'      '----------'

# the destination machine to send RTCP to. This is the address of the sender and
# is used to send back the RTCP reports of this receiver. If the data is sent
# from another machine, change this address.
DEST=192.168.1.1

# this adjusts the latency in the receiver
LATENCY=200

# the caps of the sender RTP stream. This is usually negotiated out of band with
# SDP or RTSP. normally these caps will also include SPS and PPS but we don't
# have a mechanism to get this from the sender with a -launch line.
VIDEO_CAPS="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264"
AUDIO_CAPS="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)PCMA"

VIDEO_DEC="rtph264depay ! avdec_h264"
AUDIO_DEC="rtppcmadepay ! alawdec"

VIDEO_SINK="videoconvert ! videoscale ! video/x-raw,width=720,height=480 ! autovideosink"
AUDIO_SINK="audioconvert ! audioresample ! autoaudiosink"

gst-launch-1.0 -v rtpbin name=rtpbin latency=$LATENCY                                  \
     udpsrc caps=$VIDEO_CAPS port=5000 ! rtpbin.recv_rtp_sink_0                       \
       rtpbin. ! $VIDEO_DEC ! $VIDEO_SINK                                             \
     udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0                                       \
         rtpbin.send_rtcp_src_0 ! udpsink port=5005 host=$DEST sync=false async=false\
     udpsrc caps=$AUDIO_CAPS port=5002 ! rtpbin.recv_rtp_sink_1                       \
       rtpbin. ! $AUDIO_DEC ! $AUDIO_SINK                                             \
     udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1                                       \
         rtpbin.send_rtcp_src_1 ! udpsink port=5007 host=$DEST sync=false async=false

我发现了一个非常相似的问题，如何录制而不是播放视频 stream 但我的意图是播放视频 + 音频并将两者都录制到一个文件中（例如 mp4 或 mkv）。

在另一个线程中，我认为我必须通过使用 queue、tee 和 mux 命令来执行此操作，但我需要通过正确添加它们来获得帮助。

我的想法是添加这样的命令：

...
VIDEO_DEC="rtph264depay ! avdec_h264 ! tee name=videoTee"
AUDIO_DEC="rtppcmadepay ! alawdec ! tee name=audioTee"

VIDEO_SINK="videoconvert ! videoscale ! video/x-raw,width=720,height=480 ! autovideosink !"
AUDIO_SINK="audioconvert ! audioresample ! autoaudiosink"

gst-launch-1.0 -v rtpbin name=rtpbin latency=$LATENCY                                  \
     udpsrc caps=$VIDEO_CAPS port=5000 ! rtpbin.recv_rtp_sink_0                       \
       rtpbin. ! $VIDEO_DEC ! $VIDEO_SINK                                             \
     udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0                                       \
         rtpbin.send_rtcp_src_0 ! udpsink port=5005 host=$DEST sync=false async=false\
     udpsrc caps=$AUDIO_CAPS port=5002 ! rtpbin.recv_rtp_sink_1                       \
       rtpbin. ! $AUDIO_DEC ! $AUDIO_SINK                                             \
     udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1                                       \
         rtpbin.send_rtcp_src_1 ! udpsink port=5007 host=$DEST sync=false async=false \
                   matroskamux name=mkvMux ! filesink location=test1.mkv \
                    videoTee. ! queue ! mkvMux. \
                    audioTee. ! queue ! mkvMux.

但我收到以下错误：

警告：错误的管道：无法将 autovideosink0 链接到 udpsrc1

可能我必须更改这些命令的 position 或/并且我错过了一些东西。

如果我能得到一些提示来解决这个问题，那就太好了。

Answer 1

我得到了一些专业的帮助来解决我的问题。 这是正确的方法：

VIDEO_DEC="rtph264depay ! tee name=video_splitter ! queue ! h264parse ! avdec_h264"
AUDIO_DEC="rtppcmadepay ! tee name=audio_splitter ! queue ! alawdec"

...

rtpbin. ! $AUDIO_DEC ! $AUDIO_SINK\
udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1\
rtpbin.send_rtcp_src_1 ! udpsink port=5007 host=$DEST sync=false async=false \           
matroskamux name=mux ! filesink location=test.mkv \
video_splitter. ! queue ! h264parse ! mux.video_%u \
audio_splitter. ! queue ! mux.audio_%u