简体   繁体   English

Google PubSub:如何自定义向消费者分发消息?

[英]Google PubSub : How to customize distribution of messages to consumers?

I have a scenario where we will be sending customer data to pubsub and consume it with java subscribers.我有一个场景,我们将把客户数据发送到 pubsub 并与 java 订阅者一起使用它。 I have multiple subscribers subscribed to same subscription.我有多个订阅者订阅了相同的订阅。 Is there a way to route all messages of same customerID to same subscriber ?有没有办法将相同 customerID 的所有消息路由到相同的订阅者?

I know Google Dataflow has session based windowing.我知道 Google Dataflow 有基于会话的窗口。 However, I wanted to know if we can achieve it using simple java consumers.但是,我想知道我们是否可以使用简单的 Java 消费者来实现它。

Update June 2020 : Filtering is now an available feature in Google Cloud Pub/Sub. 2020 年 6 月更新过滤现在是 Google Cloud Pub/Sub 中的一项可用功能。 When creating a subscription, one can specify a filter that looks at message attributes.创建订阅时,可以指定一个查看消息属性的过滤器。 If a message does not match the filter, the Pub/Sub service automatically acknowledges the message without delivering it to the subscriber.如果消息与过滤器不匹配,Pub/Sub 服务会自动确认该消息,而不会将其传送给订阅者。

In this case, you would need to have different subscriptions and each subscriber would consume messages from one of the subscriptions.在这种情况下,您需要有不同的订阅,并且每个订阅者都会使用来自其中一个订阅的消息。 Each subscription would have a filter set up to match the customer ID.每个订阅都将设置一个过滤器以匹配客户 ID。 If you know the list of customer IDs and it is short, you would set up an exact match filter for each customer ID, eg,如果您知道客户 ID 列表并且它很短,您可以为每个客户 ID 设置一个精确匹配过滤器,例如,

attribute.customerID = "customerID1"

If you have a lot of customer IDs and wanted to partition the set of IDs received by each subscriber, you could use the prefix operator to do so.如果您有很多客户 ID 并希望对每个订阅者收到的 ID 集进行分区,则可以使用前缀运算符来执行此操作。 For example, if the IDs are numbers, you could have filters such as:例如,如果 ID 是数字,您可以使用以下过滤器:

hasPrefix(attribute.customerID, "0")
hasPrefix(attribute.customerID, "1")
hasPrefix(attribute.customerID, "2")
hasPrefix(attribute.customerID, "3")
...
hasPrefix(attribute.customerID, "9")

Previous answer :上一个答案

At this time, Google Cloud Pub/Sub has no way to filter messages delivered to particular subscribers, no.目前,Google Cloud Pub/Sub 无法过滤传递给特定订阅者的消息,没有。 If you know a priori the number of subscribers you have, you could to it yourself, though.但是,如果您事先知道自己拥有的订阅者数量,则可以自己进行。 You could create as many topics as you have subscribers and then bucket customer IDs into different topics, publishing messages to the right topic for each customer ID.您可以创建与订阅者一样多的主题,然后将客户 ID 存储到不同的主题中,为每个客户 ID 将消息发布到正确的主题。 You'd create a single subscription on each topic and each subscriber would receive messages from one of these subscriptions.您将为每个主题创建一个订阅,每个订阅者都会收到来自这些订阅之一的消息。

The disadvantage is that if you have any subscribers that want the data for all customer IDs, then you'll have to have an additional subscription on each topic and that subscriber will have to get messages from all of those subscriptions.缺点是如果您有任何订阅者想要所有客户 ID 的数据,那么您必须对每个主题进行额外订阅,并且该订阅者必须从所有这些订阅中获取消息。

Keep in mind that you won't want to create more than 10,000 topics or else you may run up against quotas .请记住,您不会希望创建超过 10,000 个主题,否则您可能会遇到配额

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM