Convert SQL query to Elasticsearch dsl for data visualization

Question

Currently I'm using SQL queries in Elasticsearch but wanted to convert them to Elasticsearch DSL due to some data issues.

My current sql query is

{
    "query": "select \"A\",\"B\" from \"sometable\"  ORDER BY datetime DESC LIMIT 10"
}  

which returns these results:

 { "columns": [ { "name": "A", "type": "float" }, { "name": "B", "type": "float" } ], "rows": [ [ 81.22833251953125, 22.495885848999023 ], [ 81.22833251953125, 22.495885848999023 ] ] }

But I want the same result using Elasticsearch DSL in this format [[x,y],[x,y]]. I have also tried to translate the same SQL query to Elasticsearch DSL but it returns a different result.

The DSL query I have created so far is below but how we can limit the records for example 100.

{
   "aggs": {
        "documentCounts": {
          "scripted_metric": {
            "init_script": "state.transactions = []",
            "map_script": "state.transactions.add( [doc['A'].value,doc['B'].value] )",
            "combine_script": " return state.transactions"
          }
       }
    }
}

Answer 1

You could use Apache Calcite which provides an ElasticSearch adapter.

In the documentation you have an example on how to use the CSV adapter: https://calcite.apache.org/docs/tutorial.html , you can easily change the setup files to achieve the same for ES.

At that point you can use sqline to query ES via SQL statement, Calcite will do the translation to ES QL, and fetch the results.