使用 Elasticsearch 进行漏斗查询

Question

I'm trying to analyze a funnel using event data in Elasticsearch and have difficulties finding an efficient query to extract that data.

For example, in Elasticsearch I have:

timestamp          action        user id
---------          ------        -------
2015-05-05 12:00   homepage      1
2015-05-05 12:01   product page  1
2015-05-05 12:02   homepage      2
2015-05-05 12:03   checkout      1

I would like to extract the funnel statistics. For example:

homepage_count  product_page_count  checkout_count
--------------  ------------------  --------------
2             1                  1

Where homepage_count represent the distinct number of users who visited the homepage, product_page_count represents the distinct numbers of users who visited the homepage after visiting the homepage, and checkout_count represents the number of users who checked out after visiting the homepage and the product page.

What would be the best query to achieve that with Elasticsearch?

Answer 1

This can be achieved with a combination of a terms aggregation for the actions and then a cardinality sub-aggregation for the unique user count per action, like below. note that I've also added a range query in case you want to restrict the period to observe:

{
  "size": 0,
  "query": {
    "range": {
      "timestamp": {
        "gte": "2021-06-01",
        "lte": "2021-06-07"
      }
    }
  },
  "aggs": {
    "actions": {
      "terms": {
        "field": "action"
      },
      "aggs": {
        "users": {
          "cardinality": {
            "field": "user_id"
          }
        }
      }
    }
  }
}

UPDATE

This is a typical case where the scripted_metric aggregation comes in handy. The implementation is a bit naive, but it shows you the basics of implementing a funnel.

POST test/_search
{
  "size": 0,
  "aggs": {
    "funnel": {
      "scripted_metric": {
        "init_script": """
          state.users = new HashMap()
        """,
        "map_script": """
          def user = doc['user'].value.toString();
          def action = doc['action.keyword'].value;
          if (!state.users.containsKey(user)) {
            state.users[user] = [
              'homepage': false,
              'product': false,
              'checkout': false
            ];
          }
          state.users[user][action] = true;
        """,
        "combine_script": """
          return state.users;
        """,
        "reduce_script": """
          def global = [
            'homepage': 0,
            'product': 0,
            'checkout': 0
          ];
          def res = [];
          for (state in states) {
            for (user in state.keySet()) {
              if (state[user].homepage) global.homepage++;
              if (state[user].product) global.product++;
              if (state[user].checkout) global.checkout++;
            }
          }
          return global;
        """
      }
    }
  }
}

The above aggregation will return exactly the numbers you expect, ie:

  "aggregations" : {
    "funnel" : {
      "value" : {
        "product" : 1,
        "checkout" : 1,
        "homepage" : 2
      }
    }
  }