Paho MQTT while loop is blocking publish to another MQTT client

Question

My goal is proper handling of MQTT messages on Backend between some modules from IoT. I decided to implement module emulator class, which will receive my requests or send response.

Second problem is, that I need to wait for the module ACK or ERR after publish. For this issue I made ack_blocker list like this:

[
    {
        "module_mac": "mac",
        "blocked": False,
        "message": {}
    },
    {
        "module_mac": "mac",
        "blocked": False,
        "message": {}
    }
]

So when I send message to specific module, blocked attribute will be set to True and I will wait in while loop after publishing message. On the other hand my published message should come to my emulator MQTT client, where it will parse data and response ERR or ACK. On received message back, I will set blocked attribute back to False and loop will be over and return message to backend view with Error or proper message.

The problem is, that published message from backend will never arrive to emulator MQTT client. IDK why, but in my loop is timeout (10s) and after this time I should throw error that module is not responding. I was debugging whole process very carefully and when backend is going to throw error, my emulator client will finally receive message. I run that more times and it will happen exactly like that every time. So I think that loop is blocking somehow that message sending.

This is my implementation of loop:

    def send_message(self, mac: str, message: str):
        self.publish(mac, message)
        end_time = time.time() + self.timeout

        while True:
            module_ack_blocker = next(filter(lambda obj: obj.get('module_mac') == mac, self.ack_blocker), None)

            if not module_ack_blocker.get('blocked'):
                response = module_ack_blocker.get('message')

                if response.get('result') == 'OK':
                    logging.getLogger('root_logger').info(f'[MQTT]: ACK Message received.')
                    return response
                elif response.get('result') == 'ERROR':
                    raise MQTTException(response.get('details'), status_code=mqtt_status.MQTT_ERR_NOT_SUPPORTED)

            if time.time() > end_time:
                raise MQTTException('Module is not responding.', status_code=mqtt_status.MQTT_ERR_UNKNOWN)

So as you see, first I publish message. After that i will calculate timeout and loop will start. In loop I first look at proper dict in list of ack blockers (like I mentioned before). I will ask if its not blocked. And after that if I have still time for timeout.

My mqtt emulator client looks like this:

class MqttClientEmulator(object):
    def __init__(self):
        self.app = None
        self.broker_host = None
        self.broker_port = None
        self.keep_alive = None
        self.timeout = None

        self.client = mqtt.Client(client_id='brewmaster_client_emulator')

    def init(self, app):
        self.broker_host = os.getenv('BROKER_HOST')
        self.broker_port = int(os.getenv('BROKER_PORT'))
        self.keep_alive = int(os.getenv('MQTT_KEEPALIVE'))
        self.timeout = int(os.getenv('MQTT_TIMEOUT'))
        self.app = app

        self.client.on_message = self.on_message

    def on_message(self, client, userdata, msg):
        topic = msg.topic
        string_message = str(msg.payload.decode('utf-8'))
        dict_message = json.loads(string_message)

        # Request result
        if dict_message.get('device_uuid'):
            print(dict_message)
            response = {
                "module_mac": topic,
                "sequence_number": 123,
                "result": "OK",
                "details": ""
            }
            time.sleep(1)   # Just for time reserve (this code will be more complicated in future)
            self.publish('brewmaster-backend', json.dumps(response))

    def connect(self):
        self.client.connect(self.broker_host, self.broker_port, self.keep_alive)
        self.client.loop_start()

    def disconnect(self):
        self.client.loop_stop()
        self.client.disconnect()

    def subscribe(self, name):
        self.client.subscribe(name)

    def publish(self, topic, message):
        self.client.publish(topic, message)

I tried threads also and it had also no effect.

Answer 1

Ok I needed to look deeper to paho MQTT library. There is function wait_for_publish for object MQTTMessageInfo. And if you look at _condition object, it is already implemented lock with semaphores. So all I needed to was change my While loop in my MQTT client's method send_message (as shown in question) to something like this:

    def send_message(self, mac: str, message: str):
        result = self.publish(mac, message)
        end_time = time.time() + self.timeout

        if result.rc == mqtt.MQTT_ERR_QUEUE_SIZE:
            raise ValueError('Message is not queued due to ERR_QUEUE_SIZE')
        with result._condition:
            while True:
                module_ack_blocker = next(filter(lambda obj: obj.get('module_mac') == mac, self.ack_blocker), None)

                if not module_ack_blocker.get('blocked'):
                    response = module_ack_blocker.get('message')

                    if response.get('result') == 'OK':
                        logging.getLogger('root_logger').info(f'[MQTT]: ACK Message received.')
                        return response
                    elif response.get('result') == 'ERROR':
                        raise MQTTException(response.get('details'), status_code=mqtt_status.MQTT_ERR_NOT_SUPPORTED)

                if time.time() > end_time:
                    raise MQTTException('Daný modul neodpovedá.', status_code=mqtt_status.MQTT_ERR_UNKNOWN)
                result._condition.wait(1)

Where result is MQTTMessageInfo object and _condition.wait(1) is waiting with timeout 1second. So basically when is waiting all other processes are working and after 1 second will start another while loop iteration with checking, if message already arrived.