How to link Python ZMQ to a standard UDP or TCP server (legacy, written using C++ ~ 15 years ago)

Question

I need to communicate between a package written in Python (PsychoPy, for psychological / behavioral experiments) and a legacy piece of C++ software that can use UDP or TCP. In particular, on the Python / PsychoPy side I need an asynchronous process like Matlab's pnet() that will poll a socket to see if it has any data to read, process the data if yes, or just move on if no.

ZMQ was recommended to me; but all the example code I see using zmq_polling assumes that both the sending and receiving occur with ZMQ protocols. Is there some simple Python ZMQ code that connects to a non-zmq TCP or UDP source, and does polling to check for the presence of data without getting hung up if there is no data to read?

Thanks Aniruddha

import zmq
import time


# Prepare our context and sockets
context = zmq.Context()

# Bind socket to local host
receiver = context.socket(zmq.PULL)
receiver.bind("tcp://129.236.162.112:55513")
#print( "Connected to server with port %s" % port_push)

# Initialize poll set
poller = zmq.Poller()
poller.register(receiver, zmq.POLLIN)



# Process messages from socket
while True:
    print('Entered the queue')
    try:
        socks = dict(poller.poll())
    except KeyboardInterrupt:
        break

    if receiver in socks:
        message = receiver.recv()
        # process task
        print(repr(message))


    else:
        print('Nothing to show')

    time.sleep(0.01)

I can send small TCP packets from the legacy C++ machine; they get sent out without any error messages, implying no problem. But nothing happens with this Python code

The above code enters the 'try' and just stays there.

How do I access error / status messages to debug?

Thanks Aniruddha

Answer 1

_{Welcome to the Zen-of-Zero :} Yes, this is possible

Your post asks many questions at once. Let's go from one to another.

---

Q1 : How do I access error / status messages to debug?

ZeroMQ documentation presents tools for this. C-side bindings have in common to receive an explicit return code, that may get inspected via assert() plus some more details could be retrieved from errno :

void *ctx    = zmq_ctx_new();                     assert( ctx     && "EXC: Context failed to instantiate" );
void *socket = zmq_socket( ctx, ZMQ_STREAM );     assert( socket  && "EXC: Socket failed to instantiate" );
int   rc     = zmq_bind( socket, "tcp://*:8080" );assert( rc == 0 && "EXC: Bind failed to setup a Transport-Class" );

---

Q2+3 : Is there some simple Python ZMQ code that connects to a non-zmq TCP or UDP source (2) , and does polling (3) to check for the presence of data without getting hung up if there is no data to read?

For very this purpose (2) , ZeroMQ framework has been equipped somewhere about version 3.2+ with a STREAM Scalable Formal Communication Pattern Archetype. If not sure, how ZeroMQ architecture uses Context, Context's Socket(s)'s Archetypes, Socket's Transport-Class AccessPoint(s), you may like a short read into "ZeroMQ Principles in less than Five Seconds " before diving into even further details about ZeroMQ

A socket of type ZMQ_STREAM is used to send and receive TCP data from a non-ØMQ peer, when using the tcp:// transport. A ZMQ_STREAM socket can act as client and/or server, sending and/or receiving TCP data asynchronously.

When receiving TCP data, a ZMQ_STREAM socket shall prepend a message part containing the identity of the originating peer to the message before passing it to the application. Messages received are fair-queued from among all connected peers.

When sending TCP data, a ZMQ_STREAM socket shall remove the first part of the message and use it to determine the identity of the peer the message shall be routed to, and unroutable messages shall cause an EHOSTUNREACH or EAGAIN error.

To open a connection to a server, use the zmq_connect call, and then fetch the socket identity using the ZMQ_IDENTITY zmq_getsockopt call.

To close a specific connection, send the identity frame followed by a zero-length message (see EXAMPLE section).

When a connection is made, a zero-length message will be received by the application. Similarly, when the peer disconnects (or the connection is lost), a zero-length message will be received by the application.

You must send one identity frame followed by one data frame. The ZMQ_SNDMORE flag is required for identity frames but is ignored on data frames.

The use of polling (3) has two premises: never use a blocking-mode of any .recv() -methods. ZeroMQ has flags to tell the method not to block: zmq.NOBLOCK on python side. Plus, design the python code around a non-blocking form of .poll() or use a .Poller() -instance.

Example:

import zmq;                           print( zmq.zmq_version() ) # self-identify
aContext = zmq.Context();             print( "Context()", " instantiated." if zmq.zmq_errno() == 0 else " failed [#{}]".format( zmq.strerror( zmq.zmq_errno() ) ) )

aXmitSOCKET = aContext.socket( zmq.PUSH   ); aXmitSOCKET.setsockopt( zmq.LINGER, 0 ); ...
aCtrlSOCKET = aContext.socket( zmq.STREAM ); aCtrlSOCKET.setsockopt( zmq.LINGER, 0 ); ...

while True:
      if ( 0 == aXmitSOCKET.poll(  200, zmq.POLLIN ) ): # ~ 200 msec WAIT
         # ---------------------------------------------[aXmitPORT].hasNoIncomingMSG
         aCountDownREG -= 1                             #.DEC CDOWN as XmitPORT has no incoming DataToPREDICT_MSG
         aCountUpREG   += 1                             #.INC CNTUP
         if ( 0 == aCtrlSOCKET.poll( 1, zmq.POLLIN ) ): # ~   1 msec WAIT
            # ---------------------------------------------[aCtrlPORT].hasNoIncomingMSG
            ...
         else:                                          #
            # ---------------------------------------------[aCtrlPORT].hasAnIncomingMSG
            idF,aCMD = aCtrlSOCKET.recv_multipar( zmq.NOBLOCK )  # .recv()<-MSG as CtrlPORT has an incoming COMMAND_MSG
            ...
#--------------
# finally:
_ = [ aCtrlSOCKET.send_multipart( [ anIdentityFRAME, "" ], zmq.NOBLOCK ) for anIdentityFRAME in aListOfIdFRAMEs ]
aCtrlSOCKET.close()
aXmitSOCKET.close()
#--------------
# always:
aContext.term()

---

Feel free to also inspect the live-documentation of the methods:

>>> print( aCtrlSOCKET.recv_multipart.__doc__ )
Receive a multipart message as a list of bytes or Frame objects

        Parameters
        ----------
        flags : int, optional
            Any valid flags for :func:`Socket.recv`.
        copy : bool, optional
            Should the message frame(s) be received in a copying or non-copying manner?
            If False a Frame object is returned for each part, if True a copy of
            the bytes is made for each frame.
        track : bool, optional
            Should the message frame(s) be tracked for notification that ZMQ has
            finished with it? (ignored if copy=True)

        Returns
        -------
        msg_parts : list
            A list of frames in the multipart message; either Frames or bytes,
            depending on `copy`.

        Raises
        ------
        ZMQError
            for any of the reasons :func:`~Socket.recv` might fail

>>> print( aCtrlSOCKET.send_multipart.__doc__ )
Send a sequence of buffers as a multipart message.

        The zmq.SNDMORE flag is added to all msg parts before the last.

        Parameters
        ----------
        msg_parts : iterable
            A sequence of objects to send as a multipart message. Each element
            can be any sendable object (Frame, bytes, buffer-providers)
        flags : int, optional
            Any valid flags for :func:`Socket.send`.
            SNDMORE is added automatically for frames before the last.
        copy : bool, optional
            Should the frame(s) be sent in a copying or non-copying manner.
            If copy=False, frames smaller than self.copy_threshold bytes
            will be copied anyway.
        track : bool, optional
            Should the frame(s) be tracked for notification that ZMQ has
            finished with it (ignored if copy=True).

        Returns
        -------
        None : if copy or not track
        MessageTracker : if track and not copy
            a MessageTracker object, whose `pending` property will
            be True until the last send is completed.