How to combine SSL_read and epoll?

Question

I'm writing a HTTP proxy server with event-based networking library(using epoll).

Let's say there is a fast HTTPS upstream and a slow HTTP(S) downstream. So I have to store the upstream data to buffer, and detach upstream read-event from networking library when buffer is full.

But As we known, SSL_read may return data in parts. So consider the following situation:

1. buffer size is 32 and attach read-event on networking library
2. upstream send 32 bytes and the whole HTTP message is 32 bytes
3. networking library invoke read-event callback
4. call SSL_read(ssl, buf, 32) and SSL_read returns 16
5. send 16 bytes to downstream and attach read-event on networking library
6. read callback will never be invoked, because the remaining 16 bytes are in the openssl buffer and there is no more bytes from upstream

So is there a good solution?

Answer 1

OpenSSL's older SSL-based API does not follow the traditional socket I/O event model that you are trying to use.

When using a non-blocking socket, you are NOT supposed to wait for a socket read event before calling SSL_read() . You are supposed to call SSL_read() unconditionally and let it tell you when it needs more data from the socket, by returning an SSL_ERROR_WANT_READ error. Only then are you supposed to wait for a socket read event (if it returns SSL_ERROR_WANT_WRITE , wait for a socket write event instead) before calling SSL_read() again.

If you want to use traditional socket events to drive your reading/writing, use OpenSSL's newer BIO-based API instead. Use a BIO pair to push received encrypted bytes into the OpenSSL engine and let it push out decrypted bytes to you when reading, and vice versa when sending. Then you can handle the socket I/O however you want.