How to make use of materialize in Kusto?

Question

I have a function called consumption which takes 4 parameters and finally summarizes a value by Name. From this function I am trying to materialize some formulas to have as a result a better query performance.

For example:

let firstFormula = materialize (
    consumption("ID", "scale", "node", 100.0)
);

let secondformula = materialize (
    consumption("ID", "scale", "node", 60.0)
);

I am then creating a formula which divides the value coming from the firstformula by the value coming from secondformula.

let thirdFormula =  view (){
union 
    firstformula,
    secondformula
    | summarize 
        value1 = max(case(Name == "ABCD", todouble(Value), 0.0)),
        value2 = max(case(Name == "EFGH", todouble(Value), 0.0))
    | project Value = round(value1 / value2 * 100.0, 2)
};

I tested the formulas first without using the materialize function in order to check the CPU performance. I then tested it using the materialize. I ended up having a better performance from the one not using the materialize. Am I doing something wrong here? Am I using the materialize function in a wrong way?

Answer 1

The materialize() function is useful to cache query results that will be used in subsequent query statements, for example, if you have a summarization by an organization and then a column that displays it as percentage of the total, in such case materializing the results of the aggregation and then calculating the total, will reduce significantly (probably by almost a half) the processing time, for example:

let Agg = materialize(T | filter | summarize sum(Value) by Org);
let total_sum = toscalar(Agg | summarize sum(sum_Value))
Agg
| extend PercentOfTotal = sum_Value/total_sum

In Your case, it seems that you are materializing the results of processing each row but never used the materialized data multiple times afterward to justify the caching cost and this resulted in a higher query cost.