Server's NonBlocking TCP socket taking time to stream content

Question

Problem - I am working on a Streaming server & created a nonblocking socket using:

flag=fcntl(m_fd,F_GETFL);
flag|=O_NONBLOCK;
fcntl(m_fd,F_SETFL,flag);

Server then sends the Media file contents using code:

bool SendData(const char *pData,long nSize)
{
    int fd=m_pSock->get_fd();
    fd_set write_flag;
    while(1)
    {   
        FD_ZERO(&write_flag);
        FD_SET(fd,&write_flag);
        struct timeval tout;
        tout.tv_sec=0;
        tout.tv_usec=500000;

        int res=select(fd+1,0,&write_flag,0,&tout);
        if(-1==res)
        {
            print("select() failure\n");
            return false;
        }
        if(1==res)
        {
            unsigned long sndLen=0;
            if(!m_pSock->send(pData,nSize,&sndLen))
            {
                print(socket send() failure\n");
                return false;
            }
            nSize-=sndLen;
            if(!nSize)
            return true;    //everything is sent
        }
    }
}

Using above code, I am streaming a say 200sec audio file, which I expect that Server should stream it in 2-3secs using full n/w available bandwidth(Throttle off), but the problem is that Server is taking 199~200secs to stream full contents. While debugging, I commented the

m_pSock->send()

section & tried to dump the file locally. It takes 1~2secs to dump the file.

Questions - If I am using a NonBlocking TCP socket, why does send() taking so much time?

• Since the data is always available, select() will return immediately (as we have seen while dumping the file ). Does that mean send() is affected by the recv() on the client side?

Any inputs on this would be helpul. Client behavior is not in our scope.

Answer 1

Your client is probably doing some buffering to avoid network jitter, but it is likely still playing the audio file in real time. So, the file transfer rate is matched to the rate that the client is consuming the data. Since it is a 200 second audio file, it will take about 200 seconds to complete the transfer.

Answer 2

Because TCP output and input buffers are propably much smaller than the audio file, reading speed of the receiving application can slow down the sending speed.

When both the TCP output buffer of sender and the input buffer of receiver are both full, TCP stack of the sender is not able to receive any data from the sender. So sending will be blocked, until there is space.

If the receiver reads the TCP stream same speed as data is needed for playing. Then the transfer takes about 200 seconds. Or little bit less.

This can be avoided by using application layer buffering in the receiving end.

Answer 3

The problem could be that if the client side is using blocking TCP, plus is processing all the data on a single thread with no no buffer/queue etc right through to the "player" of the file, then your side being non-blocking will only speed things until you reach the point where the TCP/IP protocol stack buffers, NIC buffers etc are full. Then you will ultimately still only be able to send data as fast as the client side is consuming it. Remember TCP is a reliable, point-to-point protocol.

Where does your client code come from in your testing? Is it some sort of simple test client someone has written?