sniffing http packets using scapy

Question

Im using python 2.7.15, scapy and scapy-http on windows. I want to sniff all the http packets and extract the html pages that were sent. This is the code Im using:

from scapy.all import *
import scapy_http.http

def printPacket(packet):
    if packet.haslayer('HTTP'):
        print '='*50
        print packet.show()

sniff(prn=printPacket)

but from some reason it only captures some of the http packets(when I use the browser I dont see any packets) and I dont see any html code in the ones that it does print.

Answer 1

I think that's because some of the traffic sent is HTTPS (= HTTP + TLS). In your function you expect to HTTP application layer, which is encapsulated and encrypted in a TLS layer, and therefore it is not matched.

To sniff HTTPS, you can use this: https://github.com/tintinweb/scapy-ssl_tls (I haven't tried it yet).