Paho MQTT dynamically subscribe to new topic

Question

I am still learning how to use MQTT to publish and subscribe.

Is it possible to dynamically subscribe to the topic if the user add in new topic?

For example, the code right now is something like this:

import anotherModule

topic = anotherModule.topic #the output is similar to this -> [('Test1', 0), ('Test2', 0)]
...

def connect_mqtt() -> mqtt_client:
    def on_connect(client, userdata, flags, rc):
        if rc == 0:
            print("Connected to MQTT Broker!")
        else:
            print("Failed to connect, return code %d\n", rc)

    client = mqtt_client.Client(client_id)
    client.username_pw_set(username, password)
    client.on_connect = on_connect
    client.connect(broker, port)
    return client


def subscribe(client: mqtt_client):
    def on_message(client, userdata, msg):
        new_msg = json.loads(msg.payload.decode())
        print(new_msg)
    
    def check_topic():
        while True:
            client.subscribe(anotherModule.topic) #subscribe to the new topic if there are any
            time.sleep(1)

    t3 = threading.Thread(target=check_topic)
    t3.start()
    #  client.subscribe(topic)
    client.on_message = on_message

client = connect_mqtt()
subscribe(client)
t1 = threading.Thread(target=periodic_check_connection) #this function is irrelvant to the question
t1.start()
t2 = threading.Thread(target=client.loop_forever)
t2.start()

Right now, I am using threading to run the subscribe on another thread in a while loop. This way, it will always subscribe to any new topic if user were to add.

Are there any better ways to do this? If user were to say subscribe to 'Test3', ('Test3', 0) will be appended in the anotherModule.topic. However, mqtt is unable to subscribe to this new topic dynamically and hence, the while loop is introduced.

Any help is appreciated. Thank you!

Edit 1:

def connect_mqtt() -> mqtt_client:
    def on_connect(client, userdata, flags, rc):
        if rc == 0:
            print("Connected to MQTT Broker!")
        else:
            print("Failed to connect, return code %d\n", rc)

    client = mqtt_client.Client(client_id)
    client.username_pw_set(username, password)
    client.on_connect = on_connect
    client.connect(broker, port)
    client.subscribe(anotherModule.topic) # I tried putting here but wont work
    return client

def update_topic(new_topic):
    client.subscribe(new_topic) # How to run client.subscribe here?

def run():
    global client
    client = connect_mqtt()
    subscribe(client)
    t1 = threading.Thread(target=periodic_check_connection)
    t1.start()
    t2 = threading.Thread(target=client.loop_forever)
    t2.start()

I have tried adding global on the client but it doesnt seems to work. It says name client is not defined for client.subscribe(new_topic)

Answer 1

Yes,

You can call client.subscribe() at any time after the connection has been made and this will tell the broker to send messages for the matching pattern to that client.

You can also call client.unsubscribe() to stop receiving messages for a given topic pattern.

Calling client.subscribe() in a loop does nothing useful, it only needs to be called once per topic, subscription details are maintained by the broker not the client.

The "othermodule" should just call a function on this module that passes the topic it's interested in (and optionally a QOS) and you call client.subscribe(topic)

But it's worth pointing out that there is only 1 client.on_message callback, if you try to set this multiple times (eg each time you call subscribe) you will just overwrite the last one.

Answer 2

I think to most elegant way is to use subscription wildcards with the published topic paths being properly designed. So you might have some top-level topic Test eg and a unknown number of sub-topics like so:

Tests/Test1
Tests/Test2
Tests/Test3
...

The subscribing client only needs to subscribe to Tests/+ and will get all sub-topics automatically without making new subscriptions.

Your approach in delivering new topics within the payload of a special topic should also work but there is no need to have some endless loop subscribing. For this you might have a list managing the topics you currently have subscribed and just call client.subscribe if you got a topic name not yet existing in this list.

Answer 3

I have managed to solve this by making anotherModule a class. I have instantiate the anotherModule and pass in the client from the current module so that I can just call client.subscribe from anotherModule rather than calling the function. Thank you for your help!

# current module
def run():
    client = connect_mqtt()
    anotherModule.ModuleClass(client) #client is pass to another module
    subscribe(client)
    t1 = threading.Thread(target=periodic_check_connection)
    t1.start()
    t2 = threading.Thread(target=client.loop_forever)
    t2.start()

# anotherModule
class ModuleClass:
    def __init__(self, client):
        self.client = client
        ....

    def updateTopic(new_topic):
        self.client.subscribe(new_topic)