异步，非阻塞套接字行为 - WSAEWOULDBLOCK

Question

I have inherited two applications, one Test Harness (a client) running on a Windows 7 PC and one server application running on a Windows 10 PC. I am attempting to communicate between the two using TCP/IP sockets. The Client sends requests (for data in the form of XML) to the Server and the Server then sends the requested data (also XML) back to the client.

The set up is as shown below:

       Client                                    Server
--------------------                      --------------------  
|                  |    Sends Requests    |                  |
|   Client Socket  |  ----------------->  |   Server Socket  |
|                  |  <-----------------  |                  |
|                  |      Sends Data      |                  |
--------------------                      --------------------

This process always works on an initial connection (ie freshly launched client and server applications). The client has the ability to disconnect from the server, which triggers cleanup of sockets. Upon reconnection, I almost always (it does not always happen, but does most of the time) receive the following error:

"Receive() - The socket is marked as nonblocking and the receive operation would block"

This error is displayed at the client and the socket in question is an asynchronous, non-blocking socket.

The line which causes this SOCKET_ERROR is:

numBytesReceived = theSocket->Receive(theReceiveBuffer, 10000));

where:
- numBytesReceived is an integer (int)
- theSocket is a pointer to a class called CClientSocket which is a specialisation of CASyncSocket, which is part of the MFC C++ Library.  This defines the socket object which is embedded within the client.  It is an asynchonous, non-blocking socket.
- Receive() is a virtual function within the CASyncSocket object
- theReceiveBuffer is a char array (10000 elements)

In executing the line descirbed above, SOCKET_ERROR is returned from the function and calling theSocket->GetLastError() returns WSAEWOULDBLOCK .

SocketTools highlights that

When a non-blocking (asynchronous) socket attempts to perform an operation that cannot be performed immediately, error 10035 will be returned. This error is not fatal, and should be considered advisory by the application. This error code corresponds to the Windows Sockets error WSAEWOULDBLOCK.

When reading data from a non-blocking socket, this error will be returned if there is no more data available to be read at that time. In this case, the application should wait for the OnRead event to fire which indicates that more data has become available to read. The IsReadable property can be used to determine if there is data that can be read from the socket.

When writing data to a non-blocking socket, this error will be returned if the local socket buffers are filled while waiting for the remote host to read some of the data. When buffer space becomes available, the OnWrite event will fire which indicates that more data can be written. The IsWritable property can be used to determine if data can be written to the socket.

It is important to note that the application will not know how much data can be sent in a single write operation, so it is possible that if the client attempts to send too much data too quickly, this error may be returned multiple times. If this error occurs frequently when sending data it may indicate high network latency or the inability for the remote host to read the data fast enough.

I am consistently getting this error and failing to receive anything on the socket.

Using Wireshark , the following communications occur with the source, destinaton and TCP Bit Flags presented here:

Event: Connect Test Harness to Server via TCP/IP

Client --> Server: SYN
Server --> Client: SYN, ACK
Client --> Server: ACK

This appears to be correct and represents the Three-Way Handshake of connecting.

SocketSniff confirms that a Socket is closed on the client side.  It was not possible to get SocketSniff to work with the Windows 10 Server application.

Event: Send a Request for Data from the Test Harness

Client --> Server: PSH, ACK
Server --> Client: PSH, ACK
Client --> Server: ACK

Both request data and received data is confirmed to be exchanged successfully

Event: Disconnect Test Harness from Server

Client --> Server: FIN, ACK
Server --> Client: ACK
Server --> Client: FIN, ACK
Client --> Server: ACK

This appears to be correct and represents the Four-Way handshake of connection closure.

SocketSniff confirms that a Socket is closed on the client side.  It was not possible to get SocketSniff to work with the Windows 10 Server application.

Event: Reconnect Test Harness to Server via TCP/IP

Client --> Server: SYN
Server --> Client: SYN, ACK
Client --> Server: ACK

This appears to be correct and represents the Three-Way Handshake of connecting.

SocketSniff confirms that a new Socket is opened on the client side.  It was not possible to get SocketSniff to work with the Windows 10 Server application.

Event: Send a Request for Data from the Test Harness

Client --> Server: PSH, ACK
Server --> Client: ACK

We see no data being pushed (PSH) back to the client, yet we do see an acknowledgement.  

Has anyone got any ideas what may be going on here? I understand it would be difficult for you to diagnose without seeing the source code, however I was hoping others may have had experience with this error and could point me down the specific route to investigate.

More Info:

The Server initialises a listening thread and binds to 0.0.0.0:49720. The 'WSAStartup()', 'bind()' and 'listen()' functions all return '0', indicating success. This thread persists throughout the lifetime of the server application.

The Server initialises two threads, a read and a write thread. The read thread is responsible for reading request data off its socket and is initialised as follows with a class called Connection:

HANDLE theConnectionReadThread 
           = CreateThread(NULL,                                    // Security Attributes
                          0,                                       // Default Stacksize
                          Connection::connectionReadThreadHandler, // Callback
                          (LPVOID)this,                            // Parameter to pass to thread
                          CREATE_SUSPENDED,                        // Don't start yet
                          NULL);                                   // Don't Save Thread ID

The write thread is initialised in a similar way.

In each case, the CreateThread() function returns a suitable HANDLE, eg

theConnectionReadThread  = 00000570
theConnectionWriteThread = 00000574  

The threads actually get started within the following function:

void Connection::startThreads()
{
    ResumeThread(theConnectionReadThread);
    ResumeThread(theConnectionWriteThread);
}                                   

And this function is called from within another class called ConnectionManager which manages all the possible connections to the server. In this case, I am only concerned with a single connection, for simplicity.

Adding text output to the server application reveals that I can successfully connect/disconnect the client and server several times before the faulty behaviour is observed. For example, Within the connectionReadThreadHandler() and connectionWriteThreadHandler() functions, I am outputing text to a log file as soon as they execute.

When correct behaviour is observed, the following lines are output to the log file:

Connection::ResumeThread(theConnectionReadThread) returned 1
Connection::ResumeThread(theConnectionWriteThread) returned 1
ConnectionReadThreadHandler() Beginning
ConnectionWriteThreadHandler() Beginning

When faulty behaviour is observed, the following lines are output to the log file:

Connection::ResumeThread(theConnectionReadThread) returned 1
Connection::ResumeThread(theConnectionWriteThread) returned 1

The callback functions do not appear to being invoked.

It is at this point that the error is displayed on the client indicating that:

"Receive() - The socket is marked as nonblocking and the receive operation would block"

On the Client side, I've got a class called CClientDoc , which contains the client side socket code. It first initialises theSocket which is the socket object which is embedded within a client:

private:
    CClientSocket* theSocket = new CClientSocket;

When a connection is initialised between client and server, this class calls a function called CreateSocket() part of which is included below, along with ancillary functions which it calls:

void CClientDoc::CreateSocket()
{
    AfxSocketInit();
    int lastError;
    theSocket->Init(this);

    if (theSocket->Create()) // Calls CAyncSocket::Create() (part of afxsock.h)
    {
        theErrorMessage = "Socket Creation Successful"; // this is a CString
        theSocket->SetSocketStatus(WAITING);             
    }
    else
    {
        // We don't fall in here
    }
}

void CClientDoc::Init(CClientDoc* pDoc)
{
    pClient = pDoc; // pClient is a pointer to a CClientDoc
}

void CClientDoc::SetSocketStatus(SOCKET_STATUS sock_stat)
{
    theSocketStatus = sock_stat; // theSocketStatus is a private member of CClientSocket of type SOCKET_STATUS
}

Immediately after CreateSocket() , SetupSocket() is called which is also provided here:

void CClientDoc::SetupSocket()
{
    theSocket->AsyncSelect(); // Function within afxsock.h
}

Upon disconnection of the client from the server,

void CClientDoc::OnClienDisconnect()
{
    theSocket->ShutDown(2); // Inline function within afxsock.inl
    delete theSocket;
    theSocket = new CClientSocket;
    CreateSocket();
    SetupSocket();        
}

So we delete the current socket and then create a new one, ready for use, which appears to work as expected.

The error is being written on the Client within the DoReceive() function. This function calls the socket to attempt to read in a message.

CClientDoc::DoReceive()
{
    int lastError;
    switch (numBytesReceived = theSocket->Receive(theReceiveBuffer, 10000))
    {
    case 0:
        // We don't fall in here
        break;
    case SOCKET_ERROR: // We come in here when the faulty behaviour occurs
        if (lastError = theSocket->GetLastError() == WSAEWOULDBLOCK)
        {
            theErrorMessage = "Receive() - The socket is marked as nonblocking and the receive operation would block";
        }
        else
        {
            // We don't fall in here
        }
        break;
    default:
        // When connection works, we come in here
        break;
    }
}

Hopefully the addition of some of the code proves insightful. I should be able to add a bit more if needed.

Thanks

Answer 1

The WSAEWOULDBLOCK error DOES NOT mean the socket is marked as blocking. It means the socket is marked as non-blocking and there is NO DATA TO READ at that time.

WSAEWOULDBLOCK means the socket WOULD HAVE blocked the calling thread waiting for data if the socket HAD BEEN marked as blocking.

To know when a non-blocking socket has data waiting to be read, use Winsock's select() function, or the CClientSocket::AsyncSelect() method to request FD_READ notifications, or other equivalent. Don't try to read until there is something to read.

In your analysis, you see the client sending data to the server, but the server is not sending data to the client. So you clearly have a logic bug in your code somewhere, you need to find and fix it. Either the client is not terminating its request correctly, or the server is not receiving/processing/replying to it correctly. But since you did not show your actual code, we can't tell you what is actually wrong with it.