Efficiently listening on multiple sockets even when file descriptors are not available

Question

Let's say that I have two libraries (A and B), and each has one function that listen on sockets. These functions use select() and they return some event immediately if the data has arrived, otherwise they wait for some time ( timeout ) and then return NULL:

A_event_t* A_wait_for_event(int timeout);
B_event_t* B_wait_for_event(int timeout); 

Now, I use them in my program:

int main (int argc, char *argv[]) {
// Init A
// Init B
// .. do some other initialization
    A_event_t *evA;
    B_event_t *evB;
    for(;;) {
        evA = A_wait_for_event(50);
        evB = B_wait_for_event(50);
        // do some work based on events
    }
}

Each library has its own sockets (eg udp socket) and it is not accessible from outside.

PROBLEM : This is not very efficient. If for example there is a lot of events waiting to be delivered by *B_wait_for_event* these would have to wait always until *A_wait_for_event* timeouts, which effectively limits the throughput of library B and my program.

Normally, one could use threads to separate processing, BUT what if processing of some event require to call function of other library and vice verse. Example:

if (evA != 0 && evA == A_EVENT_1) {
    B_do_something();
}
if (evB != 0 && evB == B_EVENT_C) {
    A_do_something();
}

So, even if I could create two threads and separate functionality from libraries, these threads would have to exchange events among them (probably through pipe). This would still limit performance, because one thread would be blocked by *X_wait_for_event()* function, and would not be possible to receive data immediately from other thread.

How to solve this?

Answer 1

This solution may not be available depending on the libraries you're using, but the best solution is not to call functions in individual libraries that wait for events. Each library should support hooking into an external event loop. Then your application uses a single loop which contains a poll() or select() call that waits on all of the events that all of the libraries you use want to wait for.

glib's event loop is good for this because many libraries already know how to hook into it. But if you don't use something as elaborate as glib, the normal approach is this:

• Loop forever:
  • Start with an infinite timer and an empty set of file descriptors
  • For each library you use:
    • Call a setup function in the library which is allowed to add file descriptors to your set and/or shorten (but not lengthen) the timeout.
  • Run poll()
  • For each library you use:
    • Call a dispatch function in the library that responds to any events that might have occurred when the poll() returned.

Yes, it's still possible for an earlier library to starve a later library, but it works in practice.

If the libraries you use don't support this kind of setup & dispatch interface, add it as a feature and contribute the code upstream!

Answer 2

(I'm moving this to an answer since it's getting too long for a comment)

If you are in a situation where you're not allowed to call A_do_something in one thread while another thread is executing A_wait_for_event (and similarly for B ), then I'm pretty sure you can't do anything efficient, and have to settle between various evils.

The most obvious improvement is to immediately take action upon getting an event, rather than trying to read from both: ie order your loop

• Wait for an A event
• Maybe do something in B
• Wait for a B event
• Maybe do something in A

Other mitigations you could do are

• Try to predict whether an A event or a B event is more likely to come next, and wait on that first. (eg if they come in streaks, then after getting and handling an A event, you should go back to waiting for another A event)
• Fiddle with the timeout values to strike a balance between spin loops and too much blocking. (maybe even adjust dynamically)

  ---

EDIT: You might check the APIs for your library; they might already offer a way to deal with the problem. For example, they might allow you to register callbacks for events, and get notifications of events through the callback, rather than polling wait_for_event .

Another thing is if you can create new file descriptors for the library to listen on. eg If you create a new pipe and hand one end to library A , then if thread #1 is waiting for an A event, thread #2 can write to the pipe to make an event happen, thus forcing #1 out of wait_for_event . With the ability to kick threads out of the wait_for_event functions at will, all sorts of new options become available.

Answer 3

A possible solution is to use two threads to wait_for_events plus boost::condition_variable in "main" thread which "does something". An alike but not exact solution is here