Using Python Qpid/Proton/Messenger(), how do I filter messages from Azure Event Hubs?

Question

This gist shows how to use Messenger() to receive messages from an Azure Event Hub. It works great.

https://gist.github.com/tomconte/e2a4667185a9bf674f59

However, using this technique yields all messages in the Event Hub. I would like to read messages since an offset or timestamp (don't care which). I can see in the Qpid docs how to set these requirements, but not when using Messenger().

Here's the relevant section in the Qpid docs: https://qpid.apache.org/releases/qpid-proton-0.16.0/proton/python/api/index.html

And a sample that shows how to use it: qpid.apache.org/releases/qpid-proton-0.16.0/proton/python/examples/selected_recv.py.html

Question: is it possible and if so how?

Answer 1

Based on my understanding, I think you want to read the event data which start with an offset or timestamp on EventHub. I reviewed all classes & methods of EventHub SDK for C#/Java, then there are not any way support this usage for consuming event data from partations on EventHub. Apache Qpid is a library support AMQP protocol for Java/C/Python, and EventHub support AMQP, but it not means EventHub support all methods of Qpid.

There are two solutions as workaround way for you.

1. Receiving all messages on EventHub, and filter these undesired for you.
2. Using Azure Stream Analytics to create an output pipeline for outputing messages to other storages, such as Table Storage, DocumentDB, then you can retrieve these data from the other storages with the offset/timestamp of your needs.