Is there a library or spec for parsing complex string-based queries

Question

Have seen a pretty common pattern with some REST APIs for handling filtering by passing a composed string

One such example: https://www.algolia.com/doc/api-reference/api-parameters/filters/?language=python#examples

Example available = 1 AND (category:Book OR NOT category:Ebook) AND _tags:published AND publication_date:1441745506 TO 1441755506 AND inStock > 0 AND author:"John Doe"

Is there a good spec or parsing library for stuff like this that can take a string like this and parse it into a structure that could be then mapped to a query?

Answer 1

With any decent general parsing library you can build a parser that parse languages like this. Since it is an expression language I would suggest using some bottom-up algorithm with declarative associativity and priority spec.

For example, in parglare (disclosure: I'm the author) it would be something like:

from parglare import Grammar, Parser

grammar = r'''
S: E EOF;
E: Term | And | Not | Or | To | Parent;
And: left=E 'AND' right=E;
Or: left=E 'OR' right=E;
Not: 'NOT' val=E;
To: from=E 'TO' to=BASETYPE;
Parent: '(' E ')' {9};
Term: name=ID TermOp value=BASETYPE {10};
TermOp: ':' | '=' | '>' | '<' | '<=' | '>=';
BASETYPE: NUM | STRING | ID;

terminals
NUM: /\d+(\.\d+)?/;
ID: /[a-zA-Z_][_a-zA-Z0-9]*/;
STRING: /("(\\"|[^"])*")|(\'(\\\'|[^\'])*\')/;
'''

g = Grammar.from_string(grammar)
parser = Parser(g)

out = parser.parse(r'''
available = 1 AND (category:Book OR NOT category:Ebook)
     AND _tags:published
     AND publication_date:1441745506 TO 1441755506
     AND inStock > 0 AND author:"John Doe"
''')

out reference at the end will be a nice object model deduced from the grammar above. You can investigate it in the debugger to get the feel. I haven't covered the whole language as I don't know it but you get the idea.

It would be now relatively easy to extend this example to support the full language, and to make a mapper of the given structure to the query.

It would be even simpler if you use parglare actions to directly transform parsed input to the desired output. You can read more about it in the docs .