I hope you are loud and proud.
I'm a newbie to PyAV
and I'm using aiortc
for WebRTC MediaServer, in an aiortc
live session I have av.VideoFrame
objects available for each video frame and I want to create HLS video segments from the frames in real-time.
As you can see inthis project :
They have used OpenCV
video frame bytes piped to FFmpeg
CLI for HLS streaming
My question is, how can I use PyAV
/ python
for consuming av.VideoFrame
objects and extract 2-second video segments consisting of 60 frames for HLS streaming?
Or any python package appropriate for assembling VideoFrames to HLS stream
Thanks in advance
You are mixing some terms. I assume that HLS refers to HTTP-Streaming. WebRTC is a different way (protocol) to send video (P2P). aiortc
is not a MediaServer, even though you can do something similar. You can use aiortc
as WebRTC client that sends a video track to a browser (another WebRTC client). The connection can be established using a HTTP server as signaling server.
I assume in the following that you like to stream your video using aiortc
(WebRTC).
If I understand you correctly, your actuall question is "How to transform frames of a track (eg from webcam)?". Here is a runnable example that uses MediaStreamTrack
to implement VideoTransformTrack
. However, this example receives the webcam image from the browser (other client). Have a look at this example, which uses the webcam of your server (Python application). Just add
pc.addTrack(VideoTransformTrack(video, transform="rotate"))
to the RTCPeerConnection
( pc
in the snippet above) with video
returned from create_local_tracks
.
class MyMediaStreamTrack(MediaStreamTrack):
def __init__(self) -> None:
super().__init__()
# use webcam as source of frames in this example
self.frames = av.open(file="/dev/video0").decode()
self.kind = 'video'
async def recv(self):
# read next av.VideoFrame
frame = next(self.frames)
# optional: transform/consume frame
frame = self._transform(frame)
return frame
def _transform(self, frame):
img = frame.to_ndarray(format="bgr24")
rows, cols, _ = img.shape
M = cv2.getRotationMatrix2D((cols / 2, rows / 2), frame.time * 45, 1)
img = cv2.warpAffine(img, M, (cols, rows))
# rebuild a VideoFrame, preserving timing information
new_frame = av.VideoFrame.from_ndarray(img, format="bgr24")
new_frame.pts = frame.pts
new_frame.time_base = frame.time_base
return new_frame
pc = RTCPeerConnection()
pc.addTrack(MyMediaStreamTrack())
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.