我们知道行数据的数据缓存和组值的索引缓存。 聚合器在操作之前会处理所有要缓存的数据吗？

Question

Can you please help me to understand this by taking below example.

Group by cust_id,item_id.

what records will process to caches(index/data) in both scenarios with sorted input and unsorted input? What will be case if cache memory runs out?Which alogritham it uses to perform aggregate calculations internally?

Answer 1

I don't know about internal algorithm, but in unsorted mode, it's normal for the Aggregator to store all rows in cache and wait for the last row, because it could be the first that must be returned according to Aggregator rules ! The Aggregator will never complain about the order of incoming rows. When using cache, it will store rows first in memory, then when the allocated memory is full, it will push cache to disk. If it runs out of disk space, the session will fail (and maybe others because of that full disk). You will have to clean those files manually.

In sorted mode, there is no such problem: rows come in groups ready to be aggregated and the aggregated row will go out as soon as all rows from a group are received, which is detected when one of the values of the keys changes. The Aggregator will complain and stop if rows are not in expected order. However it pushes the problem upward to the sorting part, that could be a Sorter , which can use a lot of cache itself, or the database with an ORDER BY clause in the SQL query that could take resources on the database side.

Be careful also that SQL ORDER BY may use a different locale than Informatica.