在 aws athena 中聚合项目 sql

Question

Given a database that looks like:

item	vietnamese	cost	unique_id
fruits	trai cay	10	abc123
fruits	trai cay	8	foo99
fruits	trai cay	9	foo99
fruits	trai cay	12	abc123
fruits	trai cay	14	abc123
vege	rau	3	rr1239
vege	rau	3	rr1239

When querying through AWS Athena as such:

SELECT item, 
    sum(cost) as sum_cost, 
    avg(cost) as avg_cost, 
    array_agg(vietnamese) as vietnamese,
    array_agg(cost) as costs,
    array_agg(unique_id) as unique_ids
FROM foodtable
GROUP BY item
ORDER BY avg_cost

I'll get an array of repeated vietnamese translation:

item	vietnamese
fruits	[trai cay, trai cay, trai cay, trai cay, trai cay]

Is there a way to just keep the last/first value from the vietnamese column?

Also, with the query above, the unique_ids value would look like:

item	unique_ids
fruits	[abc123, foo99, foo99, abc123, abc123]

Is there a way to aggregate the counts and keep a counter column to achieve?

item	unique_ids
fruits	[abc123:3, foo99:2]

Currently, I've tried just reading the outputs after I get the results from the SQL query by unique-ing with set(vietnamese) and collections.Counter(unique_ids) . But if it's possible to do that in the SQL query, that'll more desirable.

Answer 1

Athena has a many functions that operate on arrays, such as filter , element_at ,cardinality , reduce , as well as functions that create and process maps . You can use these to process the aggregated arrays.

For example, to count the number of occurrences of each unique ID you can do something like this:

SELECT
  item,
  transform_values(multimap_agg(unique_id, 1), (k, v) -> cardinality(v))
GROUP BY item