python 3 : how to wait for the callback in async function when use asyncio tasks

Question

AWS IoT python Sdk has a function publishAsync
the signature of the function is publishAsync(topic, payload, QoS, ackCallback=None)

I want to warp this to a coroutine and use asyncio.

what I want to do is this:

1. connect to AWS
2. publish 10 messages asynchronously
3. disconnect

I don't know how should I wrap that function to a async function.

async def asyncPublish(self,
                     msg:str,
                     topic:str,
                     QoS=1):

        # the publishAckFn
        def internalPubAckHandler(mid):
            print(json.dumps({
                'messageID':mid,
                'acknowledged':True
            }))
            return True
        pass

        # publish to the topic asynchronously
        messageID = self.awsIoTClient.publishAsync(topic,msg,QoS,ackCallback=internalPubAckHandler)
        print(json.dumps({
           'messageID':messageID,
           'topic':topic,
           'payload':msg,
           'QoS':QoS,
           'async':True
        }))

    pass


---- in my main file
tasks = []
for i in range(10):
        tasks += asyncPublish('test','test')
pass

loop = asyncio.get_event_loop()
loop.set_debug(True)  # debug mode
gw1.connect()
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
gw1.disconnect()  

but when I do this, the disconnect will be called sooner than internalPubAckHandler and I will not get any acknowledgment.

Answer 1

Finally, I find the solution. We can use futures.

Future objects are used to bridge low-level callback-based code with high-level async/await code. Ref

If the signature of the function is something like this:

def functionWithCallback(args,callback)

and we need to wrap this to a async function that waits for the callback, we can do something like the following:

async def asyncVersion(args):

    # one way of creating a future
    loop = asyncio.get_running_loop()
    future = loop.create_future()

    def futureCallback():
        # do something here and then signal the future
        ...
        loop.call_soon_threadsafe(future .set_result,time.time())
    pass

    # call your function
    functionWithCallback(args,futureCallback)

    # wait for the callback
    await future


    # do other stuff after the callback is done
    .....

pass