How to understand where the UDP packets are dropped in my ubuntu C program

Question

I am using a python tool to generate packets in one VM and I capture them in my program running as a linux process in another VM. Both the VMs are ubuntu and they are running on the same subnet. I notice that some of the packets get dropped in my program. What is the best tool to know where the packets are dropped?

I see that RcvbufErrors in netstat output gets increased as I send new packets.

# netstat -us
IcmpMsg:
    InType0: 14
    InType3: 1493
    InType5: 204
    InType8: 54
    InType13: 5
    InType17: 5
    OutType0: 54
    OutType3: 645946
    OutType8: 584
    OutType14: 5
Udp:
    7686124 packets received
    646545 packets to unknown port received.
    33928069 packet receive errors
    7157259 packets sent
    RcvbufErrors: 33928069
    IgnoredMulti: 345772
UdpLite:
IpExt:
    InMcastPkts: 4
    InBcastPkts: 363522
    InOctets: 13243409806
    OutOctets: 8445992434
    InMcastOctets: 144
    InBcastOctets: 114457552
    InNoECTPkts: 100191590
    InECT0Pkts: 143

Answer 1

Well, if you think that you need to find out where some packets get dropped, this might not be useful since you already know that at least RcvbufErrors figure is growing. This means that the NIC is able to deliver the packets to the kernel but the latter is unable to deliver the packets to the application, obviously, because a fixed-size receive buffer gets full faster than the application could read the data out (or just flush it).

So, I'd say such a result just gives an impression of poor application design. Perhaps, you should consider some good technique to enhance packet capture in your application so that more packets are captured/examined/dropped per second. A good example is PACKET_MMAP Rx ring which has an exhaustive description and is widely used in libpcap -based applications. The latter aspect makes it indeed reasonable to use wireshark or just tcpdump instead of your hand-made app to inspect the real capture rate, etc.