Hive如何在count（*）中使用Max？

Question

I have two tables :

Fly : Year , Origin

Airport : Code , Name

Here is a sample of data :

Fly :

1989,SF    
1989,SF   
1989,NY  
1993,NY  
1998,Par     
1998,Par  
1998,NY

AirPort :

SF, International Airport    
NY, Inter Air    
Par, Charles de Gaulle

I want to get the most used airport per year.

So firstly I did this request to get ths number of occurence of each airport per each year :

SELECT v.Year,a.airport ,count(*)
From airports a JOIN Vol v ON (a.iata = v.Dest)
Group By v.Year,a.airport
ORDER BY Year ASC,airport ASC;

So i get this kind of result :

1989, San Francisco, 2  
1989, New York, 1
1993, New York, 1
1998, new York, 1
1998, Paris, 2

And I want the max of each year like this :

1989, San Francisco, 2
1993, New York, 1
1998, Paris, 2

Can i do it with one single request ? Should i use an intermediate table ?

Is it better with Pig ?

Thank you in advance

Answer 1

This is a little tricky in Hive, but certainly doable. It requires two things: using your first query as an subquery for a bigger one, and a little trick to do an "arg-max".

SELECT Year, max(named_struct('n', n, 'airport', airport)) FROM (
  SELECT v.Year, a.airport, count(*) as n
  FROM airports a JOIN Vol v ON (a.iata = v.Dest)
  GROUP BY v.Year, a.airport
) t
GROUP BY Year;

Notice that named_struct creates a struct field, and those compare in order of their first field first, so you get the correct "max" behavior while still retaining the airport name. This does mean that your output will be in the form of a struct, though:

1989, {n:2, airport:San Francisco}
1993, {n:1, airport:New York}
1998, {n:2, airport:Paris}

If you want to "un-struct" it, you just need to select out those fields individually:

SELECT Year, max(named_struct('n', n, 'airport', airport)).n, max(named_struct('n', n, 'airport', airport)).airport FROM (
  SELECT v.Year, a.airport, count(*) as n
  FROM airports a JOIN Vol v ON (a.iata = v.Dest)
  GROUP BY v.Year, a.airport
) t
GROUP BY Year;