如何处理 MQTT 订阅者中的部分消息？

Question

I have an IoT gateway device that publishes data to a MQTT broker every minute. However, the message comes as several packets. I need to recombine these partial message packets into one complete message, parse it and then store in a database. I'm using paho python MQTT library. The problem is that the callback is triggered upon each received partial packet but I want to trigger it only when all packets associated with the same message are received. Is there anyway to do that? Cheers

Sample partial messages look like this:

Packet 1 =

{'DB3': {'CH1-01': '243.7300', 'CH1-02': '0.0000', 'CH1-03': '0.0000', 'CH1-04': '243.7300', 'CH1-05': '0.0000', 'CH1-06': '243.7300', 'CH1-07': '0.0000', 'CH1-08': '0.0000', 'CH1-09': '0.0000', 'CH1-10': '0.0440', 'CH1-11': '0.0000', 'CH1-12': '0.0000', 'CH1-13': '0.0440', 'CH1-14': '-0.0040', 'CH1-15': '0.0000', 'CH1-16': '0.0000', 'CH1-17': '-0.0040', 'CH1-18': '0.0440', 'CH1-19': '0.0000', 'CH1-20': '0.0000', 'CH1-21': '0.0440', 'CH1-22': '0.0000', 'CH1-23': '0.0000', 'CH1-24': '0.0000', 'CH1-25': '0.0000', 'CH1-26': '50.0160', 'CH1-27': '3.6400', 'CH1-28': '0.0000', 'CH1-29': '0.0000', 'CH1-30': '0.4000', 'CH1-31': '3.6400', 'CH1-32': '0.0000', 'CH1-33': '0.0000', 'CH1-34': '0.4000', 'CH1-35': '0.0000', 'CH1-36': '0.0000', 'CH1-37': '0.0000', 'CH1-38': '0.0000', 'CH1-39': '0.0000', 'CH1-40': '0.0000', 'CH1-41': '0.0000', 'CH1-42': '0.0000', 'CH2-01': '243.6100', 'CH2-02': '0.0000', 'CH2-03': '0.0000', 'CH2-04': '243.6100', 'CH2-05': '0.0000', 'CH2-06': '243.6100', 'CH2-07': '0.0000', 'CH2-08': '0.0000'}, 'ts': 1637040150000}

Packet 2 =

{'DB3': {'CH2-09': '0.0000', 'CH2-10': '0.0000', 'CH2-11': '0.0000', 'CH2-12': '0.0000', 'CH2-13': '0.0000', 'CH2-14': '0.0000', 'CH2-15': '0.0000', 'CH2-16': '0.0000', 'CH2-17': '0.0000', 'CH2-18': '0.0040', 'CH2-19': '0.0000', 'CH2-20': '0.0000', 'CH2-21': '0.0040', 'CH2-22': '0.0000', 'CH2-23': '0.0000', 'CH2-24': '0.0000', 'CH2-25': '0.0000', 'CH2-26': '50.0200', 'CH2-27': '0.0000', 'CH2-28': '0.0000', 'CH2-29': '0.0000', 'CH2-30': '0.0000', 'CH2-31': '0.0000', 'CH2-32': '0.0000', 'CH2-33': '0.0000', 'CH2-34': '0.0000', 'CH2-35': '0.0000', 'CH2-36': '0.0000', 'CH2-37': '0.0000', 'CH2-38': '0.0000', 'CH2-39': '0.0000', 'CH2-40': '0.0000', 'CH2-41': '0.0000', 'CH2-42': '0.0000', 'CH3-01': '243.6400', 'CH3-02': '0.0000', 'CH3-03': '0.0000', 'CH3-04': '243.6400', 'CH3-05': '0.0000', 'CH3-06': '243.6400', 'CH3-07': '0.0000', 'CH3-08': '0.0000', 'CH3-09': '0.0000', 'CH3-10': '0.0000', 'CH3-11': '0.0000', 'CH3-12': '0.0000', 'CH3-13': '0.0000', 'CH3-14': '0.0000', 'CH3-15': '0.0000', 'CH3-16': '0.0000'}, 'ts': 1637040150000}

Answer 1

This isn't a really a MQTT or Paho question, both are working exactly as expected, the problem is with the device publishing the messages if there is no way to determine how many parts it will send or the start/end of a stream of packets.

My best suggestion is to set a timeout after the first message and when that expires assemble all the messages that arrived in that period. What you set that timeout to you'll have to determine, but if you have configured the data to come in bursts every 60 seconds then possibly start at 15 seconds.