由单词和分隔符组成的分层字符串的近似字符串匹配

Question

I'm looking for a data structure which supports matching strings against a set of patterns where the strings represent mqtt topics . The strings are defined to be composed of words ("topic level") separated by a slash character. Examples for strings would be "topic1/topic2" or "//topic1/topic2" which contains an empty topic level. The character set is UTF-8 excluding '#' and '+'.

Patterns are topic strings but can contain two wildcards. The first wildcard character "#" can only be used at the end of a pattern and matches an arbitrary number of following topics, ie "a/#" matches any strings where "a/" is a prefix. The second pattern "+" matches a single arbitrary topic. For example, “sport/tennis/+” matches “sport/tennis/player1” and “sport/tennis/player2”, but not “sport/tennis/player1/ranking”. Also, because the single-level wildcard matches only a single level, “sport/+” does not match “sport” but it does match “sport/”.

The use-case is that clients register for interesting topics providing a pattern. When a message is sent, it is published with a topic string. The string has to be matched against registered subscribers, so I am looking for a data structure that efficiently (in terms of space and time) selects the subscribers whose registered patterns match the published topic.

I was thinking about using a suffix tree or trie because this would allow fast prefix matches when "#" is used. The nodes in the trie would contain the subscribers for this string, and a set of all subscribers of sub-strings. This should allow quick look-ups for exact and prefix queries, but I don't know if this supports the "+" wildcard.

Another approach I am thinking of is to create a directed graph where each node contains one topic and an edge topic1 -> topic2 if there is a sub-string "topic1/topic2" in a pattern. With this graph, I could traverse the nodes topic by topic. A "+" wildcard would just mean to traverse to all children.

An obvious alternative are regular expressions which would result in a finite state-machine which is probably similar to the graph approach. However, I was hoping to find something faster.

The algorithm should be used in a mqtt broker where subscribers can register and deregister topics any time, so it must also support updating the search data structure by adding or removing patterns.

Answer 1

Aho-corasick finite-state-machine supports wildcards. You can also reverse a trie and search for wildcards: http://phpir.com/tries-and-wildcards/