Handling SSL_shutdown correctly

Question

The OpenSSL documentation on SSL_shutdown states that:

It is therefore recommended, to check the return value of SSL_shutdown() and call SSL_shutdown() again, if the bidirectional shutdown is not yet complete (return value of the first call is 0).

https://www.openssl.org/docs/ssl/SSL_shutdown.html

I have a code snippet below where I check for return value 0 from SSL_shutdown and call it again, which I have been using. My question is, is it okay to disregard the return value of SSL_shutdown on the second call or we should keep retrying SSL_shutdown until a 1 (bidirectional shutdown complete) is returned.

int r = SSL_shutdown(ssl);
//error handling here if r < 0 
if(!r)
{
    shutdown(fd,1);
    SSL_shutdown(ssl); //how should I handle return value and error handling here is it required?? 
}
SSL_free(ssl);
SSLMap.erase(fd);
shutdown(fd,2);
close(fd);

Answer 1

openssl is a bit of a dark art.

Firstly the page you referenced has HTML-ified the return values badly. Here's what the man-page actually says:

  RETURN VALUES

   The following return values can occur:

   0   The shutdown is not yet finished. Call SSL_shutdown() for a second
       time, if a bidirectional shutdown shall be performed.  The output
       of SSL_get_error(3) may be misleading, as an erroneous
       SSL_ERROR_SYSCALL may be flagged even though no error occurred.

   1   The shutdown was successfully completed. The "close notify" alert
       was sent and the peer's "close notify" alert was received.

   -1  The shutdown was not successful because a fatal error occurred
       either at the protocol level or a connection failure occurred. It
       can also occur if action is need to continue the operation for non-
       blocking BIOs.  Call SSL_get_error(3) with the return value ret to
       find out the reason.

If you have blocking BIOs, things are relatively simple. A 0 on the first call means you need to call SSL_shutdown again if you want a full bidirectional shutdown. Basically it means that you sent a close_notify alert but haven't one back yet). A 1 would mean you previously received a close_notify alert from the other peer, and you're totally done. A -1 means an unrecoverable error. On the second call (which you only do if you got a 0 back), then a bidirectional shutdown is initiated (ie now wait from the other side for them to send you their "close_notify" alert). Logic dictates you can't get a 0 back again (because it's a blocking BIO and will have completed the first step). A -1 indicates an error, and a 1 indicates completion success.

If you have non-blocking BIOs, the same "possibly 0 then 1" return values apply, save for the fact you need to go through the whole SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE rigmarole as well, ie:

   If the underlying BIO is non-blocking, SSL_shutdown() will also return
   when the underlying BIO could not satisfy the needs of SSL_shutdown()
   to continue the handshake. In this case a call to SSL_get_error() with
   the return value of SSL_shutdown() will yield SSL_ERROR_WANT_READ or
   SSL_ERROR_WANT_WRITE. The calling process then must repeat the call
   after taking appropriate action to satisfy the needs of SSL_shutdown().
   The action depends on the underlying BIO. When using a non-blocking
   socket, nothing is to be done, but select() can be used to check for
   the required condition. When using a buffering BIO, like a BIO pair,
   data must be written into or retrieved out of the BIO before being able
   to continue.

So you have two levels of repetition. You call SSL_shutdown the 'first' time but repeat if you get SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE after going around the select() loop in the normal way, and only count the 'first' SSL_shutdown as done if you get a non SSL_ERROR_WANT_ error code (in which case it failed), or you get a 0 or 1 return. If you get a 1 return, you've done. If you get a 0 return, and you want a bidirectional shutdown, then you have to do the second call, on which again you will need to check for SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE and retry select; that should not return 1 , but may return 0 or an error.

Not simple.

Couple more notes from the docs : after calling SSL_shutdown and getting a "0" back the first time, you could optionally then call SSL_read instead of SSL_shutdown (in case the peer is still sending you any data on that SSL socket), and, I guess, "hope" that they eventually send you a close message from their side, to flush the pipes.

Also if you're planning on closing the socket after shutdown completion "anyway" you could entirely skip the second call to SSL_shutdown (the "1" of the "0 then 1") and just go ahead and close the socket, the kernel should take care of discarding the "now ignored" close_notify alert that presumably they should be about to send...