How do I implement non-blocking socket receives with ZeroMQ in Erlang or Elixir?

Question

In Python I have the option of using a "poller" object which polls blocking sockets for messages waiting and unblocks after a specified number of milliseconds (in the case below, 1000, in the while True block):

import zmq

# now open up all the sockets
context = zmq.Context()
outsub = context.socket(zmq.SUB)
outsub.bind("tcp://" + myip + ":" + str(args.outsubport))
outsub.setsockopt(zmq.SUBSCRIBE, b"")
inreq = context.socket(zmq.ROUTER)  
inreq.bind("tcp://" + myip + ":" + str(args.inreqport))
outref = context.socket(zmq.ROUTER)  
outref.bind("tcp://" + myip + ":" + str(args.outrefport))
req = context.socket(zmq.ROUTER)  
req.bind("tcp://" + myip + ":" + str(args.reqport))
repub = context.socket(zmq.PUB)  
repub.bind("tcp://" + myip + ":" + str(args.repubport))

# sort out the poller
poller = zmq.Poller() 
poller.register(inreq, zmq.POLLIN)
poller.register(outsub, zmq.POLLIN)
poller.register(outref, zmq.POLLIN)
poller.register(req, zmq.POLLIN)

# UDP socket setup for broadcasting this server's address 
cs = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
cs.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
cs.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)

# housekeeping variables
pulsecheck = datetime.utcnow() + timedelta(seconds = 1)
alivelist = dict()
pulsetimeout = 5

while True: 
    polls = dict(poller.poll(1000))
    if inreq in polls:
        msg = inreq.recv_multipart()
        if msg[1] == b"pulse":           # handle pluse
            ansi("cyan", False, textout = " pulse" + "-" + msg[0].decode())
            if not msg[0] in alivelist.keys():
                handlechange(msg[0])
            alivelist[msg[0]] = datetime.utcnow() + timedelta(seconds = pulsetimeout)
    if outsub in polls:
        msgin = outsub.recv_multipart()[0]
        repub.send(msgin) # republish
        msg = unpacker(msgin)
        if isinstance(msg, dict):
            valu = msg.get("value")
            print(".", end = "", flush = True)
        else:
            ansi("green", False, textout = msg)

    if req in polls:
        msg = req.recv_multipart()
        valmsg = validate_request(msg)
        if not valmsg[0]:
            ansi("red", True); print(valmsg[1]); ansi()
        elif len(alivelist) > 0:
            targetnode = random.choice(list(alivelist.keys()))
            inreq.send_multipart([targetnode, packer(valmsg[1])])
            ansi("blue", True, textout = "sent to " + targetnode.decode())
        else:
            ansi("red", True, textout = "NO CONNECTED NODES TO SEND REQUEST TO")
    if outref in polls:
        msg = outref.recv_multipart()
        destinataire, correlid = msg[1].split(b"/")
        req.send_multipart([destinataire, correlid, msg[2]])

I want to implement something analogous in Elixir (or Erlang) but my preferred native library, chumak , doesn't seem to implement polling. How do I implement non-blocking receives in Erlang/Elixir, preferably using Chumak, but I'll move to another Erlang zeroMQ library if necessary? My socket pattern preference is router sends, dealer receives.

EDIT

My use case is the following. I have a third party financial service which serves data based on requests, with answers coming asynchronously. So you can send multiple requests, and you'll get responses back after an unspecified period of time, and not necessarily in the same order you sent them.

So I need to connect this service into Erlang (actually Elixir) and ZeroMQ seems like a good fit. Multiple users connected (via Phoenix) to Erlang/Elixir will send requests, and I need to pass these on to this service.

The problem comes if there is an error in one of the requests, or the third party service has some kind of problem. I will be blocking-waiting for a response, and then unable to service new requests from Phoenix.

Basically I want to listen constantly for new requests, send them over, but if one request doesn't produce a response, I will have one-fewer responses than requests and that will lead to an eternal wait.

I understand that if I send requests separately, then the good ones will produce responses so I don't need to worry about blocking even if, over time, I get quite a big numerical difference between requests sent and responses received. Maybe the design idea is that I shouldn't worry about this? Or should I try to track one-for-one responses to requests and timeout the non-responses somehow? Is this a valid design pattern?

Answer 1

Is your system constantly connected to the asynchronous query resource, or are you making a new connection with each query?

Each situation has its own natural model in Erlang.

The case of: A single (or pool of) long-term connection(s)

Long-term connections that maintain a session with the resource (the way a connection with a database would work) are most naturally modelled as processes within your system that have the sole job of representing that external resource.

The requirements of that process are:

• Translate the external resource's messages into internally meaningful messages (not just passing junk through -- don't let raw, external data invade your system unless it is totally opaque to you)
• Keep track of timed out requests (and this may require something sort of like polling, but can be done more precisely with erlang:send_after/3

This implies, of course, that the module that implements this process will need to speak the protocol of that resource. But if this is accomplished then there really isn't any need for a messaging broker like an MQ application.

This allows you to have that process be reactive and block on receive while the rest of your program goes off to do whatever its doing to do. Without some arbitrary polling that will surely run you into the Evil Black Swamp of Scheduling Issues.

The case of: A new connection per query

If each query to the resource requires a new connection the model is similar, but in here you spawn a new process per query and it represents the query itself within your system. It blocks waiting for the response (on a timeout), and nothing else matters to it.

That is the easier model, actually, because then you don't have to scrub a list of past, possibly timed out requests that will never return, don't have to interact with a set of staged timeout messages sent via erlang:send_after/3 , and you move your abstraction one step closer to the actual model of your problem .

You don't know when these queries will return, and that causes some potential confusion -- so modeling each actual query as a living thing is an optimal way to cut through the logical clutter.

Either way, model the problem naturally: As a concurrent, asynch system

In no case, however, do you want to actually do polling the way you would in Python or C or whatever. This is a concurrent problem, so modelling it as such will provide you a lot more logical freedom and is more likely to result in a correct solution that lacks corners that give rise to weird cases.