如何设置 aiohttp https 服务器和客户端？

Question

I'm trying to learn how I might secure data from being altered after being passed over an open.network between a server and a worker

in my head I was thinking that it should follow something like:

|server|---send_job----->|worker|
|      |<--send_results--|      |
|      |                 |      |
|      |-send_kill_req-->|      |

obviously I don't want someone altering my send_job to do something nefarious, and I don't want someone to be peeking at my results.

so I have a super simple aiohttp client/server setup that I'm trying to implement ssl into but I'm completely lost.

below is the most basic stuff I've tried, but I've also tried implementing my own ssl certificates via:

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout domain_srv.key -out domain_srv.crt

along with following the documentation but I'm still never able to get any response at all.

How would I properly implement the ssl_context to get this to work?!

server.py

from aiohttp import web
import msgpack
import ssl

async def handle(request):
    name = request.match_info.get('name', "Anonymous")
    text = "Hello, " + name
    return web.Response(text=text)

app = web.Application()
app.add_routes([web.get('/', handle),
                web.get('/{name}', handle)])

ssl_context = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
web.run_app(app, ssl_context=ssl_context)

client.py import aiohttp import asyncio import ssl

async def main():
    sslcontext = ssl.create_default_context(purpose=ssl.Purpose.CLIENT_AUTH)
    async with aiohttp.ClientSession() as session:
        async with session.get("https://0.0.0.0:8443", ssl=sslcontext) as response:
            html = await response.read()
            print(html)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

1. run python3 server.py in one window
2. run python3 client.py in another window

I then end up usually with something like:

Traceback (most recent call last):
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/site-packages/aiohttp/connector.py", line 822, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/asyncio/base_events.py", line 804, in create_connection
    sock, protocol_factory, ssl, server_hostname)
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/asyncio/base_events.py", line 830, in _create_connection_transport
    yield from waiter
ConnectionResetError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "client.py", line 14, in <module>
    loop.run_until_complete(main())
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
    return future.result()
  File "client.py", line 9, in main
    async with session.get("https://0.0.0.0:8443", ssl=sslcontext) as response:
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/site-packages/aiohttp/client.py", line 843, in __aenter__
    self._resp = await self._coro
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/site-packages/aiohttp/client.py", line 366, in _request
    timeout=timeout
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/site-packages/aiohttp/connector.py", line 445, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/site-packages/aiohttp/connector.py", line 757, in _create_connection
    req, traces, timeout)
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/site-packages/aiohttp/connector.py", line 879, in _create_direct_connection
    raise last_exc
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/site-packages/aiohttp/connector.py", line 862, in _create_direct_connection
    req=req, client_error=client_error)
  File "/home/mEE/miniconda3/envs/py3/lib/python3.6/site-packages/aiohttp/connector.py", line 829, in _wrap_create_connection
    raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 0.0.0.0:8443 ssl:<ssl.SSLContext object at 0x7fe4800d2278> [None]

---

Solution:

This was a two part problem,

1. I had no idea what I was doing with openssl, the requests library helped me figure this out!

    import requests requests.get("https://0.0.0.0:8443", verify="domain_srv.crt") SSLError: HTTPSConnectionPool(host='0.0.0.0', port=8443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname '0.0.0.0' doesn't match None",),))

   As it turns out, those lines I just made default when making my openssl certificates actually mattered. A slightly more correct (but probably still wrong) config similar to

   Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:. Locality Name (eg, city) []:. Organization Name (eg, company) [Inte.net Widgits Pty Ltd]:. Organizational Unit Name (eg, section) []:. Common Name (eg server FQDN or YOUR name) []:0.0.0.0 Email Address []:.

   led me to the result:

    import requests requests.get("https://0.0.0.0:8443", verify="domain_srv.crt") SubjectAltNameWarning: Certificate for 0.0.0.0 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)

   It would appear that 'subjectAltName' is something a little more difficult to add, requiring a lot more work than a simple command, you'll want to follow a guide like this , I will try it and see if that error goes away.

2. I think I was using ssl.Purpose.CLIENT/SERVER_AUTH wrongly, as @Andrej mentioned, I switched that around (as I will show below) and made a few other changes and now I am getting the correct responses. I'll just say that, I definitely still don't understand ssl.Purpose but at least I have something that I can work with for now, and hopefully I'll figure out the rest in time.
   client.py

    import aiohttp import asyncio import ssl async def main(): sslcontext = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH, cafile='domain_srv.crt') async with aiohttp.ClientSession() as session: async with session.get("https://0.0.0.0:8443/JOHNYY", ssl=sslcontext) as response: html = await response.read() print(html) loop = asyncio.get_event_loop() loop.run_until_complete(main())

   server.py

    from aiohttp import web import ssl async def handle(request): name = request.match_info.get('name', "Anonymous") text = "Hello, " + name return web.Response(text=text) app = web.Application() app.add_routes([web.get('/', handle), web.get('/{name}', handle)]) ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) ssl_context.load_cert_chain('domain_srv.crt', 'domain_srv.key') web.run_app(app, ssl_context=ssl_context)

   commandline

    >python3 server.py # Switch to a new window/pane >python3 client.py b'Hello, JOHNYY'

EDIT : I just wanted to post an update for anyone that's working on this type of problem. I think that using the python cryptography library is a nicer way of generating the crt/key files so if you're interested feel free to use/modify this template (I make no promise that these are best practices):

#!/usr/bin/env python
"""
stuff for network security
"""

import socket
import datetime

from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography import x509
from cryptography.x509.oid import NameOID
from cryptography.hazmat.primitives import hashes

import attr


@attr.s(auto_attribs=True)
class Netsec:
    hostname: str = attr.Factory(socket.gethostname)
    out_file_name: str = "domain_srv"

    def generate_netsec(self):
    # GENERATE KEY
        key = rsa.generate_private_key(
            public_exponent=65537,
            key_size=2048,
            backend=default_backend(),
        )

        with open(f"{self.out_file_name}.key", "wb") as f:
            f.write(key.private_bytes(
                encoding=serialization.Encoding.PEM,
                format=serialization.PrivateFormat.TraditionalOpenSSL,
                encryption_algorithm=serialization.NoEncryption(),
            ))

        subject = issuer = x509.Name([
            x509.NameAttribute(NameOID.COUNTRY_NAME, u"US"),
            x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, u"CA"),
            x509.NameAttribute(NameOID.LOCALITY_NAME, u"Wala Wala"),
            x509.NameAttribute(NameOID.ORGANIZATION_NAME, u"A place"),
            x509.NameAttribute(NameOID.COMMON_NAME, self.hostname),
        ])

        cert = x509.CertificateBuilder().subject_name(
            subject
        ).issuer_name(
            issuer
        ).public_key(
            key.public_key()
        ).serial_number(
            x509.random_serial_number()
        ).not_valid_before(
            datetime.datetime.utcnow()
        ).not_valid_after(
            # Our certificate will be valid for 5 years
            datetime.datetime.utcnow() + datetime.timedelta(days=365*5)
        ).add_extension(
            x509.SubjectAlternativeName([
                x509.DNSName(u"localhost"),
                x509.DNSName(self.hostname),
                x509.DNSName(u"127.0.0.1")]),
            critical=False,
        # Sign our certificate with our private key
        ).sign(key, hashes.SHA256(), default_backend())

        with open(f"{self.out_file_name}.crt", "wb") as f:
            f.write(cert.public_bytes(serialization.Encoding.PEM))

Answer 1

You are creating the certificates but not loading them to the SSL chain. And change your ssl_context creation from ssl.Purpose.SERVER_AUTH to ssl.Purpose.CLIENT_AUTH :

from aiohttp import web
import ssl

async def handle(request):
    name = request.match_info.get('name', "Anonymous")
    text = "Hello, " + name
    return web.Response(text=text)

app = web.Application()
app.add_routes([web.get('/', handle),
                web.get('/{name}', handle)])


ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_context.load_cert_chain('domain_srv.crt', 'domain_srv.key')

web.run_app(app, ssl_context=ssl_context)

When you run your server, the client will print upon connection:

b'Hello, Anonymous'

Answer 2

Maybe useful for beginners like myself: If you run across the "IP address mismatch, certificate is not valid for..' problem after implementing the above solution, search for how to create a self signed certificate for IP addresses with alternative names.