Low UDP throughput comparing to iperf UDP performance

Question

When measuring UDP throughput between Windows PC and Zynq-based device by iperf2 tool, I am getting around 950 Mb/s over dedicated 1Gb Ethernet link. However, when using my own UDP application on PC I am getting only around 50 Mb/s, which is drastically lower throughput than measured by iperf. Of course, in my UDP application I don't have any processing, only while loop in which I am calling sendto function, with UDP packets of sizes of 1470 bytes. Application on Zynq device is provided by XAPP1026, so it's not mine. I am looking at iperf code trying to figure out what they do differently, but basically, I can't find any socket or udp options or anything similar they do in order to maximize UDP throughput.

Here is the code of the main function (MAXUDP define is 1470):

int main(int argc, char** argv) 
{
int sockfd;
struct sockaddr_in servaddr;
char sendline[MAXUDP];
int i;
int j;
const int tr_size = ( 200 * MB );
const int npackets = ( tr_size / MAXUDP );
const int neval = 2;
DWORD start;
DWORD end;
int optval;

WSADATA wsaData;
if(WSAStartup(MAKEWORD(2, 1), &wsaData) != 0 )
{
    printf( "Err: %d\n", WSAGetLastError() );
    exit(1);
}

bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons(SERV_PORT);
servaddr.sin_addr.s_addr = inet_addr("172.16.0.215");

sockfd = Socket(AF_INET, SOCK_DGRAM, 0);
Connect(sockfd, (const SA*) &servaddr, sizeof(servaddr));

optval = 208*KB;
Setsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, (const char*) &optval, sizeof optval);

prep_data(sendline, MAXUDP);

for ( i = 1; i <= neval; i++ )
{
    start = GetTickCount();
    for ( j = 0; j < npackets/neval; j++ )
        sendto(sockfd, sendline, MAXUDP, 0, NULL, NULL);
    end = GetTickCount() - start;

    printf("Time elapsed: %d sec.\n", end/1000);
    printf("Throughput: %d.%3d MB/s\n", (tr_size/neval)/end/1000, (tr_size/neval)/end - (tr_size/neval)/end/1000);
}
return 0;
}

So, my main question is how to maximize UDP throughput in the same way iperf does it?

UPDATE: I switched to Ubuntu PC. Results are different, but still there is some random stuff happening. The first thing I do is set IP addresses for eth0 ( ifconfig eth0 172.16.0.200 netmask 255.255.255.0 ) and gateway address ( route add default gw 172.16.0.1 ). When I run iperf with iperf -c 172.16.0.215 -i 5 -t 25 -u -b 1000m ) I am getting around 800 Mbits/sec. However, after few runs of iperf in same way, all of sudden I start getting only around 15 Mbits/sec or even much less. I figured out that I need to set IP, netmask and gateway addresses once again in order to get 800 Mbits/sec. Also, my UDP application behaves in the same way. I am measuring 957 Mbits/sec (with MAXUDP set to 1470) after I had run commands for setting IP addresses. But after few iterations it slows down to around 11 Mbits/sec. Then I set IP addresses again, and the behavior repeats itself. So, as Kariem stated in his answer, the problem is not in the code itself, bur rather in some OS, netif configuration related stuff. However, I must run my UDP application on Windows, so I need to figure out what is happening there. If you guys have any ideas on what could be happening in Windows, please let me now.

Answer 1

You have a mistake in the way you're calculating throughput. Your size is set in bytes, so you're calculating your throughput in bytes while iperf does it in bits.

change

printf("Throughput: %d.%3d MB/s\n", (tr_size/neval)/end/1000, (tr_size/neval)/end - (tr_size/neval)/end/1000);

to this

printf("Throughput: %d.%3d MB/s\n", ((tr_size/neval)/end/1000)*8, (tr_size/neval)/end - ((tr_size/neval)/end/1000)*8);

I ran a version of your code in my machine and I'm getting 1GB throughput. Here it is

#include <netinet/in.h>
#include <string.h>
#include <sys/time.h>
#include <cstdio>
#include <arpa/inet.h>
#include <fcntl.h>

#define MAXUDP 1470
#define SERV_PORT 5001
static inline long int getCurTimeInMs()
{
    struct timeval tp;
    gettimeofday(&tp, NULL);
    return tp.tv_sec * 1000 + tp.tv_usec / 1000;

}


int main(int argc, char** argv)
{
    int sockfd;
    struct sockaddr_in servaddr;
    char sendline[MAXUDP];
    int i;
    int j;
    const int tr_size = ( 10000 * 1024*1024 );
    const int npackets = ( tr_size / MAXUDP );
    const int neval = 2;

    int optval;

    bzero(&servaddr, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_port = htons(SERV_PORT);

        servaddr.sin_addr.s_addr = inet_addr("10.0.1.2");

    sockfd = socket(AF_INET, SOCK_DGRAM, 0);


    connect(sockfd, (const sockaddr*) &servaddr, sizeof(servaddr));


    optval = 208*1024;
    setsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, (const char*) &optval, sizeof optval);
    long int start = 0, end = 0;
    for ( i = 1; i <= neval; i++ )
    {
        start = getCurTimeInMs();
        for ( j = 0; j < npackets/neval; j++ )
            sendto(sockfd, sendline, MAXUDP, 0, NULL, NULL);
        end = getCurTimeInMs() - start;

        printf("Time elapsed: %d sec.\n", end/1000);
        printf("Throughput: %d.%3d MB/s\n", (tr_size/neval)/end/1000 * 8, (tr_size/neval)/end - (tr_size/neval)/end/1000);

    }
    return 0;
}

Answer 2

Why did you define MAXUDP 1470? Try setting it to 65535, measure again and report here.

You should not be confusing ethernet frame size (1500 bytes) with UDP datagram size. Different things. Let the IP stack do the necessary fragmentation instead of your application. That may be more efficient.