TCP-Streaming: Audio/Video Synchronization

Question

I have a question regarding TCP/Live-Streaming/Synchronization (programming language: Java):

Let's say I have a Client, that is capturing the Audio and Video from a camera. It is sending the audio and video data simultaneously (via Threads) via TCP-sockets to a server, that is receiving the audio and video stream simultaneously on two different threads.

[I know, UDP is normally used for live-streaming applications but there are reasons, why I'm using TCP..]

So, the question is: Will there be any chance to get out of sync ever via TCP? There are two streams on different sockets (one for audio, one for video), there may be a problem, that a packet has to be re-sent, but will the resend of the lost-packet be a problem (I mean, when resending a packet via TCP, will this cause a permanent delay or will the client just send the lost packet and the next packet faster for a few milliseconds)?

I'm sending the audio and video signal at the same time to the server. Will they always arrive at the same time on the server? If not, is there a way to produce/simulate the problem?

I've done some tests and the audio and video was always in sync. But I'm not sure, if it will always be like that (even after more than 5 hours of streaming).

Thank you.

Answer 1

If you use two different sockets, there's no guarantee. TCP guarantees data transmission in the correct order, but if you use two different streams for audio and video, it doesn't apply.

However, if the connection is fast enough there shouldn't be a problem. It sends it as fast as it can. If one stream gets "clogged up" though, it will delay and fall out of sync, but I doubt that would happen with your case for sending videos (as long as it's not too high an FPS or resolution).

I think it would be safer to use a single stream, but I think your way is fine to be honest. It all depends on: speed of the client, speed of the server, and speed of the network.