Efficient way to capture packets in multithreading Python program

Question

I'm implementing a sniffer using Python. I need in all packet details (VLAN etc.) so I'm using RAW sockets. There are several sniffers working as demons on all of my Ethernet interfaces, so each one is in a different thread.

Is there any better way to get the results in my main thread besides global variable to which I add my packets (as in the code below)?

I tried Queue but didn't see any particular benefits besides added complexity.

import socket, threading

def ReceivePackets():
    soc = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0003))  
    while 1:
        packet = soc.recv(2000)  
        buffer.append(packet)

def ReadSniffer():
    result = list(buffer) 
    #Clear buffer to start sniffing from scratch after reading
    del buffer[:]
return result

I start sniffers on all of my Ethernet interfaces at the beginning of the program and read the corresponding global buffers when I need the data.

buffer = []
t = threading.Thread(target = ReceivePackets)
t.daemon = True
t.start()
...
packets = ReadSniffer()

Answer 1

With sockets I always got inconsistent results, since I have various packets with different packet size. A socket may capture all the packets I send few times, but eventually it loses some packets.

I moved to pcapy interface to libcap library instead of raw sockets and it works perfectly and very reliably.

I implemented Sniffer class as a subclass of threading.Thread so it starts in a separate thread. The implementation is like this:

class Sniffer(threading.Thread):

    def __init__(self, port):      
        ...
        threading.Thread.__init__(self)
        #Packets will be stored here
        self.result = []
        #Not worry about closing the sniffer thread
        self.daemon = True    

    #Invoked when I start my thread
    def run(self):      
        max_bytes = 16000
        #Important to capture broken packets as I need them
        promiscuous = True
        read_timeout = 100
        pc = pcapy.open_live(str(self.port), max_bytes, promiscuous, read_timeout)
        pc.loop(-1, self.recv_pkts)

    #Do when the packet arrives
    def recv_pkts(self, hdr, packet):
        packetHex = binascii.hexlify(packet)
        self.result.append(packetHex)
        return self.result

To start sniffer thread in parallel to your main program:

 sniffer1 = Sniffer(eth1)
 #Start separate thread
 sniffer1.start()