non-blocking i/o using select

Question

I have a sample client-server program that does non-blocking I/O for several sockets not using processes or threads. It uses select . Unfortunately, the server just shows lots of blank lines and that's all. Where is the mistake?
Running on MacOS.

Thanks in advance.

Server:

import socket
import select

sock = socket.socket()
sock.bind(('', 10001))
sock.listen()

conn1, _ = sock.accept()
conn2, _ = sock.accept()

conn1.setblocking(0)
conn2.setblocking(0)

epoll = select.poll()
epoll.register(conn1.fileno(), select.POLLIN | select.POLLOUT)
epoll.register(conn2.fileno(), select.POLLIN | select.POLLOUT)

conn_map = {
    conn1.fileno(): conn1,
    conn2.fileno(): conn2,
}

while True:
    events = epoll.poll(1)
    for fileno, event in events:
        if event & select.POLLIN:
            data = conn_map[fileno].recv(1024)
            print(data.decode('utf8'))
        elif event & select.POLLOUT:
            conn_map[fileno].send('ping'.encode('utf8'))

Client:

import socket
from multiprocessing import Pool

def create_socket_and_send_data(number):
    with socket.create_connection(('127.0.0.1', 10001)) as sock:
        try:
            sock.sendall(f'client {number}\n'.encode('utf8'))
        except socket.error as ex:
            print('data sending error', ex)

    print(f'data for {number} has been sent')

if __name__ == '__main__':
    with Pool(processes=2) as pool:
        pool.map(create_socket_and_send_data, range(2))

Answer 1

Unfortunately, the server just shows lots of blank lines and that's all.

Actually this is not true. The server prints at the beginning the lines it got from the clients. After they've send these lines the client close the connection which means that select.POLLIN gets triggered again on the socket and recv returns empty data.

This empty data is the sign that the peer has closed the connection. Once it got this sign the server should close the connection to the client and remove the fileno from the select . Instead your server prints the empty string with a newline and continues to expect new POLLIN events. These will come again and again and will always an empty buffer, thus leading to all the empty lines you see.

Answer 2

select is paradoxically easier to use for input than for output. For input, you receive an event each time new data arrives on a socket, so you always ask for all the sockets and have something to process for every new event.

For output, select will just say that a socket if ready to accept new data. Which is almost always true except if you have just filled a buffer. So you should only poll for an output socket when you have something to write there.

So you should register your sockets with select.POLLIN only. For the write part, you should either directly write to a socket without polling if you can hope that the peer should always be able to receive, or set up a queue with pending output per socket, modify the polling state of a socket with select.POLLIN | select.POLLOUT select.POLLIN | select.POLLOUT when there is something in its queue and modify it back with select.POLLIN back when the queue is empty again.