Python Queue and Threading confusion

Question

I made a sniffer in Python which calculates the size of IP packets. I tried to get the size in two ways:

1) simply by calculating len(pkt)

2) by extracting the packet length from the IP header and adding 14bytes for the ethernet header

When comparing the result from len(pkt) with the extracted value from the IP header, they were almost always the same (ok, for very few packets there was a difference of 4-6 bytes but that's another question).

But once I implemented queues and threading into my code, the sizes from len(pkt) and the extracted value from the IP header are in most cases totally different. Sometimes there is a difference of just few bytes, and sometimes several hundred bytes. But very rarely they are the same.

The code in which I implemented threading is below. Does anybody have an idea if I made a mistake in the way I implemented threading/queues or what I am doing wrong?

import pcap
import struct
import dpkt
from Queue import Queue
from threading import Thread

def packet_handler():
    ts,pkt=q.get()
    eth=dpkt.ethernet.Ethernet(pkt)
    if eth.type != dpkt.ethernet.ETH_TYPE_IP:
        return
    a=struct.unpack('!BBHHHBBH4s4s', pkt[14:34])
    print a[2]+14,len(pkt)

def start():  
    pc.loop(0,lambda ts,pkt: q.put((ts,pkt)))

q=Queue()
pc=pcap.pcap(name="eth0")
start_sniffer=Thread(target=start)
start_sniffer.start()

while True:
    packet_handler()

Part of the output looks like this:

419 1454
419 419
54 60
389 60
389 389
405 60
405 405
405 60
405 405
54 60
54 60
493 491
491 492
491 493
491 491
502 502
54 60

Answer 1

I think I solved it. I changed this line:

pc.loop(0,lambda ts,pkt: q.put((str(ts),str(pkt))))

That means, I converted ts and pkt to strings before adding them to the queue.

Sizes for few packets are still different for 4-6 bytes, but that was also the case with the not-threading version. I will probably ask about that in a new post.

Thanks everybody for the help!!!