How to handle ssl connections in raw Python socket?

Question

I'm writing a program to download a given webpage. I need to only use raw python sockets for all the connection due to some restriction. So I make a socket connection to a given domain (the Host field in the response header of an object) and then send the GET request on this. Now when the url is a https url, I think I need to first do the SSL handshake (because otherwise I'm getting non-200 OK responses from the server and other error responses mentioning P3P policies). I inspected curl's response to check how it's able to successfully download while I'm not, turns out curl first does the SSL handshake (that's all the difference). curl is always able to successfully download a given object, the only difference always being the SSL handshake it does.

So I'm wondering how to do the SSL handshake in raw python sockets? Basically I want as easy a solution which allows me to do the minimum besides using raw sockets.

Answer 1

Here is an example of a TCP client with SLL.

Not sure if it's the best way to download a web page but it should answer your question "SSL handshake in raw python socket".

You will probably have to adapt the struct.pack/unpack but you get the general idea:

import socket
import ssl
import struct
import binascii
import sys

class NotConnectedException(Exception):
    def __init__(self, message=None, node=None):
        self.message = message
        self.node = node

class DisconnectedException(Exception):
    def __init__(self, message=None, node=None):
        self.message = message
        self.node = node

class Connector:
    def __init__(self):
        pass

    def is_connected(self):
        return (self.sock and self.ssl_sock)

    def open(self, hostname, port, cacert):
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.ssl_sock = ssl.wrap_socket(self.sock, ca_certs=cacert, cert_reqs=ssl.CERT_REQUIRED)

        if hostname == socket.gethostname():
            ipaddress = socket.gethostbyname_ex(hostname)[2][0]
            self.ssl_sock.connect((ipaddress, port))
        else:
            self.ssl_sock.connect((hostname, port))

        self.sock.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)

    def close(self):
        if self.sock: self.sock.close()
        self.sock = None
        self.ssl_sock = None

    def send(self, buffer):
        if not self.ssl_sock: raise NotConnectedException("Not connected (SSL Socket is null)")
        self.ssl_sock.sendall(struct.pack('L', len(buffer)))
        self.ssl_sock.sendall(buffer)

    def receive(self):
        if not self.ssl_sock: raise NotConnectedException("Not connected (SSL Socket is null)")
        data_size_buffer = self.ssl_sock.recv(4)

        if len(data_size_buffer) <= 0:
            raise DisconnectedException()

        data_size = struct.unpack('L', data_size_buffer)[0]
        received_size = 0
        data_buffer = ""

        while received_size < data_size:
            chunk = self.ssl_sock.recv(1024)
            data_buffer += chunk
            received_size += len(chunk)

        return data_buffer

Then you use the class like this:

    connector = Connector.Connector()
    connector.open(server_ip, server_port, path_to_the_CA_cert.pem)
    connector.send(your_data)
    response = connector.receive()
    connector.close()

Answer 2

You can use the wrap_socket method of the python ssl module to turn your socket into one that talks SSL. Once you've done this you can use it like normal, but internally the data will be encrypted and decrypted for you. These are the docs for the method: https://docs.python.org/2/library/ssl.html#ssl.wrap_socket

Answer 3

I think the easier way to do that would be using SSL contexts and wraping the TCP socket.

Python SSL module's documentation give a very thoroughful explanation with examples. I recommend you to read the relevant sections of Python 2 or Python 3 ssl module documentation. It should be very easy to achieve what you want.

Hope this helps!